Screen-to-screen communication via touch sense elements

ABSTRACT

A method includes a first computing device generating a signal having an oscillating component and driving the signal on a first touch sense element of the first computing device. The method continues with the first computing device detecting a touch on the first touch sense element based on the signal. While the touch is detected, the method continues by the first computing device modulating the signal with data to produce a modulated data signal. The method continues with a second computing device receiving the modulated data signal via a transmission medium and a second touch sense element of the second computing device, where the transmission medium includes at least one of a human body and a close proximity between the first and second computing devices. The method continues with by the second computing device demodulating the modulated data signal to recover the data.

CROSS REFERENCE TO RELATED PATENTS

The present U.S. Utility patent application claims priority pursuant to35 U.S.C. § 119(e) to U.S. Provisional Application No. 62/970,306,entitled “Screen-to-Screen Communication Network and ApplicationsThereof”, filed Feb. 5, 2020, which is hereby incorporated herein byreference in its entirety and made part of the present U.S. Utilitypatent Application for all purposes.

STATEMENT REGARDING FEDERALLY SPONSORED RESEARCH OR DEVELOPMENT

Not Applicable.

INCORPORATION-BY-REFERENCE OF MATERIAL SUBMITTED ON A COMPACT DISC

Not Applicable.

BACKGROUND OF THE INVENTION Technical Field of the Invention

This invention relates generally to data communication systems and moreparticularly to data communication between touch screens.

Description of Related Art

A computing device is known to communicate data, process data, and/orstore data. The computing device may be a cellular phone, a laptop, atablet, a personal computer (PC), a work station, a video game device, aserver, and/or a data center that support millions of web searches,stock trades, or on-line purchases every hour.

A computing device may also transmit data to another computing devicevia a near proximity communication. For example, a computing device mayuse near field communication (NFC), infrared (IR), and/or Bluetooth (BT)to communicate data over short distances. In some examples, the use ofnear proximity communications are utilized for point-of-sale (POS)transactions and other data communications where security of the data isdesired.

BRIEF DESCRIPTION OF THE SEVERAL VIEWS OF THE DRAWING(S)

The patent or application file contains at least one drawing executed incolor. Copies of this patent or patent application publication withcolor drawing(s) will be provided by the Office upon request and paymentof the necessary fee.

FIG. 1 is a schematic block diagram of an embodiment of a communicationsystem in accordance with the present invention;

FIG. 2 is a schematic block diagram of an embodiment of a computingdevice in accordance with the present invention;

FIG. 3 is a schematic block diagram of another embodiment of a computingdevice in accordance with the present invention;

FIG. 4 is a schematic block diagram of another embodiment of a computingdevice in accordance with the present invention;

FIG. 5 is a schematic block diagram of another embodiment of a computingdevice in accordance with the present invention;

FIG. 6 is a schematic block diagram of another embodiment of a computingdevice in accordance with the present invention;

FIG. 7 is a schematic block diagram of another embodiment of a computingdevice in accordance with the present invention;

FIG. 8 is a schematic block diagram of another embodiment of a computingdevice in accordance with the present invention;

FIG. 9 is a schematic block diagram of an embodiment of a touch screendisplay in accordance with the present invention;

FIG. 10 is a schematic block diagram of an embodiment of a touch screenin accordance with the present invention;

FIG. 11 is a schematic block diagram of an embodiment of a drive sensemodule in accordance with the present invention;

FIG. 12A is a schematic block diagram of an embodiment of a drive sensecircuit in accordance with the present invention;

FIG. 12B is a schematic block diagram of another embodiment of a drivesense circuit in accordance with the present invention;

FIG. 13 is a schematic block diagram of an embodiment of drive sensemodules in accordance with the present invention;

FIG. 14 is a schematic block diagram of another embodiment of a usercomputing device and an interactive computing device in accordance withthe present invention;

FIG. 15 is a schematic block diagram of an embodiment of ascreen-to-screen (STS) connection in accordance with the presentinvention;

FIG. 16 is a schematic block diagram of another embodiment of ascreen-to-screen (STS) connection in accordance with the presentinvention;

FIG. 17 is a schematic block diagram of an embodiment of another examplea screen-to-screen (STS) connection in accordance with the presentinvention;

FIG. 18 is a schematic block diagram of an embodiment of an example offorming multiple screen to screen (STS) connections in accordance withthe present invention;

FIG. 19 is a schematic block diagram of an embodiment of another examplean example of forming multiple screen to screen (STS) connections inaccordance with the present invention;

FIG. 20 is a schematic block diagram of an embodiment of an example oftransmitting close proximity signals in accordance with the presentinvention;

FIG. 21 is a schematic block diagram of an embodiment of another exampleof transmitting close proximity signals in accordance with the presentinvention;

FIG. 22 is a logic flow diagram of an example of a method fordetermining which type of communication to use in accordance with thepresent invention;

FIG. 23 is a logic flow diagram of an example of a method of a first andsecond computing device communicating via a screen to screen (STS)connection in accordance with the present invention;

FIG. 24 is a schematic block diagram of an embodiment of a computingdevice in accordance with the present invention;

FIG. 25 is a schematic block diagram of an embodiment of a communicationin accordance with the present invention;

FIG. 26 is a schematic block diagram of another embodiment of an exampleof a communication in accordance with the present invention;

FIG. 27 is a schematic block diagram of another embodiment of an exampleof a communication in accordance with the present invention;

FIG. 28 is a schematic block diagram of another embodiment of an exampleof a communication in accordance with the present invention;

FIG. 29 is a schematic block diagram of another embodiment of an exampleof a communication in accordance with the present invention;

FIG. 30 is a schematic block diagram of another embodiment of an exampleof a communication in accordance with the present invention;

FIG. 31 is a logic flow diagram of an example of a method of determininga type of communication to use for an interaction in accordance with thepresent invention;

FIG. 32 is a schematic block diagram of an embodiment of an embodimentof initiating and setting up screen to screen (STS) communications inaccordance with the present invention;

FIG. 33 is a logic flow diagram of another example of a method ofsetting up a screen to screen (STS) communications in accordance withthe present invention;

FIG. 34 a logic flow diagram of another example of a method of settingup a screen to screen (STS) communications in accordance with thepresent invention;

FIG. 35 is a schematic block diagram of an embodiment of an example oftransmitting close proximity signals in accordance with the presentinvention;

FIG. 36 is a schematic block diagram of an embodiment of an example oftransmitting ping signals in accordance with the present invention;

FIG. 37 is a schematic block diagram of an embodiment of an example ofan interactive computing device (ICD) 12 generating a default pingsignal in accordance with the present invention;

FIG. 38 is a schematic block diagram of an embodiment of an example of adefault ping signal in accordance with the present invention;

FIG. 39 is a schematic block diagram of an embodiment of an example of adefault ping signal in accordance with the present invention;

FIG. 40 is a schematic block diagram of another embodiment of an exampleof transmitting a default ping signal in accordance with the presentinvention;

FIG. 41 is a logic flow diagram of an example of a method for setting upa screen to screen connection in accordance with the present invention;

FIG. 42 is a schematic block diagram of an embodiment of affectedelectrodes of an interactive computing device in accordance with thepresent invention;

FIG. 43 is a schematic block diagram of an example of receiving adefault ping signal in accordance with the present invention;

FIG. 44 is a schematic block diagram of another embodiment of receivinga ping signal in accordance with the present invention;

FIG. 45 is a schematic block diagram of an embodiment of an example ofgenerating a ping back signal in accordance with the present invention;

FIG. 46 is a schematic block diagram of an embodiment of an example ofproducing a ping back signal in accordance with the present invention;

FIG. 47 is a logic flow diagram of an example of a method of setting upa screen to screen (STS) connection in accordance with the presentinvention;

FIG. 48 is a logic flow diagram of another example of a method for usein setting up a screen to screen (STS) connection in accordance with thepresent invention;

FIG. 49 is a logic flow diagram of another example of a method ofsetting up a screen to screen (STS) connection in accordance with thepresent invention;

FIG. 50 is a schematic block diagram of an embodiment of an example of aradio frequency (RF) transceiver and a signal source in accordance withthe present invention;

FIG. 51 is a schematic block diagram of an embodiment of an interactivecomputing device (ICD) interacting with a user computing device (UCD) toselect items in accordance with the present invention;

FIG. 52 is a schematic block diagram of an embodiment of an example ofan interactive computing device (ICD) interacting with a user computingdevice (UCD) to mirror a menu of items in accordance with the presentinvention;

FIG. 53 is a schematic block diagram of an embodiment of an example ofan interactive computing device (ICD) interacting with a user computingdevice (UCD) to select items of a menu in accordance with the presentinvention;

FIG. 54 is a schematic block diagram of another embodiment of an exampleof an interactive computing device (ICD) interacting with a usercomputing device (UCD) to edit a menu selection in accordance with thepresent invention;

FIG. 55 is a logic flow diagram of an example of an interactivecomputing device (ICD) interacting with a user computing device (UCD) toedit a menu selection in accordance with the present invention;

FIG. 56 is a schematic block diagram of an example of an interactivecomputing device (ICD) interacting with a user computing device (UCD) toedit a menu selection in accordance with the present invention;

FIG. 57 is a schematic block diagram of an example of an interactivecomputing device (ICD) interacting with a user computing device (UCD) toedit a menu selection in accordance with the present invention;

FIG. 58 is a schematic block diagram of an example of an interactivecomputing device (ICD) interacting with a user computing device (UCD) toedit a menu selection in accordance with the present invention;

FIG. 59 is a schematic block diagram of an embodiment of setting upscreen to screen (STS) communications in accordance with the presentinvention;

FIG. 60 is a schematic block diagram of an embodiment of the setting upscreen to screen communications in accordance with the presentinvention;

FIG. 61 is a schematic block diagram of the example of the setting upthe screen to screen (STS) communications in accordance with the presentinvention;

FIG. 62 is a schematic block diagram of an embodiment of the example ofthe setting up screen to screen (STS) communications in accordance withthe present invention;

FIG. 63 is a logic flow diagram of an example of a method of determininga menu interaction modality in accordance with the present invention;

FIG. 64 is a logic flow diagram of an example of a method of setting upa screen to screen (STS) communication in accordance with the presentinvention;

FIG. 65 is a schematic block diagram of an embodiment of processing ascreen to screen transaction in accordance with the present invention;

FIG. 66 is a logic flow diagram of an example of a method of processinga screen to screen transaction in accordance with the present invention;

FIG. 67 is a logic flow diagram of another example of a method ofprocessing a screen to screen transaction in accordance with the presentinvention;

FIG. 68 is a schematic block diagram of an embodiment of processing ascreen to screen transaction in accordance with the present invention;

FIG. 69 is a schematic block diagram of an embodiment of the processinga screen to screen transaction in accordance with the present invention;

FIG. 70 is a logic flow diagram of an example of a method of setting upa screen to screen connection in accordance with the present invention;and

FIG. 71 is a logic flow diagram of an example of a method of setting upa screen to screen connection in accordance with the present invention.

DETAILED DESCRIPTION OF THE INVENTION

FIG. 1 is a schematic block diagram of an embodiment of a communicationsystem 10 that includes a plurality of interactive computing devices 12,a personal private cloud 13, a plurality of user computing devices 14,networks 15, a cloud service host device 16, a plurality of interactionapplication servers 20, a plurality of screen-to-screen (STS)communication servers 22, a plurality of payment processing servers 24,an independent server 26 and a database 27. In an embodiment, computingdevices 12-14 include a touch screen with sensors and drive-sensemodules. In another embodiment, computing devices 12-14 include a touch& tactic screen that includes sensors, actuators, and drive sensemodules. In yet another embodiment, computing devices 12-14 include atouch sensor with a display and/or a display without a touch screen.

The computing devices 12 and 14 may each be a portable computing deviceand/or a fixed computing device. A portable computing device may be asocial networking device, a gaming device, a cell phone, a smart phone,a digital assistant, a digital music player, a digital video player, alaptop computer, a handheld computer, a tablet, a video game controller,and/or any other portable device that includes a computing core. A fixedcomputing device may be a computer (PC), a computer server, a cableset-top box, a point-of-sale equipment, interactive touch screens, asatellite receiver, a television set, a printer, a fax machine, homeentertainment equipment, a video game console, and/or any type of homeor office computing equipment.

An interactive computing device 12 performs screen-to-screen (STS)communications with a user computing device 14 via an STS wirelessconnection 18. Although not explicitly shown, the STS wirelessconnection may be formed between two or more ICDs and/or two or moreUCDs. The term wireless indicates the communication is performed atleast in part without a wire. For example, the STS wireless connectionis via a transmission medium (e.g., one or more of a human body, closeproximity (e.g., within a few inches), a surface (for vibrationencoding, etc.). In an embodiment, the STS wireless connection 18 isperformed via a local direct communication (e.g., not performed vianetwork 15). The STS wireless connection 18 may be in accordance with adata protocol (e.g., data format, encoding parameters, frequency range,etc.), which will be discussed in further detail with reference to oneor more subsequent figures.

The interactive computing device 12 also stores data that enables a userand/or a user computing device to use and/or interact with theinteractive computing device in a variety of ways. For example, thestored data includes system applications (e.g. operation system, etc.),user applications (e.g., restaurant menus, etc.), payment processingapplications, etc. The data may be stored locally (e.g., within theinteractive computing device) and/or externally (e.g., within one ormore interaction application servers, etc.).

A user computing device 14 is also operable to perform screen-to-screen(STS) communications with one or more other user computing devices 14and/or interactive computing devices 12 via an STS wireless connection18. The user computing device 14 also stores data to enable a user touse the computing device in a variety of ways. For example, the storeddata includes system applications (e.g., operating system, etc.), userapplications (e.g., word processing, email, web browser, etc.), personalinformation (e.g., contact list, personal data), and/or paymentinformation (e.g., credit card information etc.). The data may be storedlocally (e.g., within the computing device) and/or externally. Forinstance, at least some of the data is stored in a personal privatecloud 13, which is hosted by a cloud service host device 16. As aspecific example, a word processing application is stored in a personalaccount hosted by the vendor of the word processing application. Asanother specific example, payment information for a credit card isstored in a private account hosted by the credit card company and/or bythe vendor of the computing device. The computing devices 12-14 will bediscussed in greater detail with reference to one or more subsequentfigures.

A server 20-26 is a type of computing device that processes largeamounts of data requests in parallel. A server 20-26 includes similarcomponents to that of the computing devices 12 and 14 with more robustprocessing modules, more main memory, and/or more hard drive memory(e.g., solid state, hard drives, etc.). Further, a server 20-26 istypically accessed remotely; as such it does not generally include userinput devices and/or user output devices. In addition, a server 20-26may be a standalone separate computing device and/or may be a cloudcomputing device.

The screen-to-screen (STS) communication server 22 supports andadministers STS communications between UCDs and ICDs. For instance, theSTS communication server 22 stores an STS communication application thatmay be installed and/or run on the user computing device 14 and theinteractive computing device 12. As a specific example, the STScommunication server is a cellular provider server (e.g., Verizon,T-Mobile, etc.). In an example, a user of a user computing device 14registers with the STS communication server 22 to install and/or run theSTS communication application on the user computing device 14. The UCDand/or the ICD may utilize a cellular connection (e.g., network 15) todownload the STS communication application. In an embodiment, the STScommunication server 22 functions to perform a patch distribution of theSTS application for the interactive computing device 12 via an agreementbetween the interactive application server 20 and STS communicationserver 22.

The interaction application server 20 supports transactions between aUCD and an ICD that are communicating via an STS wireless connection.For example, the UCD using its user interaction application to interfacewith the ICD to buy items at a coffee shop and the ICD accesses itsoperator interaction application to support the purchase. In addition,the UCD (e.g., cell phone of a user) and/or ICD (e.g., POS device of acoffee shop) accesses the interaction application server to retrievepersonal preferences of the user. (e.g., likes weather information,likes headlines news, ordering preferences, etc.). The transaction iscompleted via the STS wireless connection.

The payment processing server 24 stores information on one or more ofcardholders, merchants, acquirers, credit card networks and issuingbanks in order to process transactions in the communication network. Forexample, a payment processing server 24 is a bank server that storesuser information (e.g., account information, account balances, personalinformation (e.g., social security number, birthday, address, etc.),etc.) and user card information for use in a transaction. As anotherexample, a payment processing server is a merchant server that storesgood information (e.g., price, quantity, etc.) and may also storecertain user information (e.g., credit card information, billingaddress, shipping address, etc.) acquired from the user.

The independent server 26 stores publicly available data (e.g., weatherreports, stock market information, traffic information, public socialmedia information, etc.). The publicly available data may be free or maybe for a fee (e.g., subscription, one-time payment, etc.). In anexample, the publicly available data is used in setting up an STScommunication. For example, a tag in a social media post associated witha user of the UCD initiates an update check to interactive applicationsinstalled on the UCD that are associated with nearby companies. Thisensures STS communications are enabled on the UCD for a more seamlessSTS transaction when the user is ready to transmit data via an STSconnection. As another example, when a user is en route to a restaurant,weather information and traffic information are utilized to determine anestimated time to place a pre-order for one or more menu items from therestaurant that is to be completed (e.g., paid for, authorize a payment,etc.) utilizing an STS wireless connection.

A database 27 is a special type of computing device that is optimizedfor large scale data storage and retrieval. A database 27 includessimilar components to that of the computing devices 12 and 14 with morehard drive memory (e.g., solid state, hard drives, etc.) and potentiallywith more processing modules and/or main memory. Further, a database 27is typically accessed remotely; as such it does not generally includeuser input devices and/or user output devices. In addition, a database27 may be a standalone separate computing device and/or may be a cloudcomputing device.

The network 15 includes one more local area networks (LAN) and/or one ormore wide area networks (WAN), which may be a public network and/or aprivate network. A LAN may be a wireless-LAN (e.g., Wi-Fi access point,Bluetooth, ZigBee, etc.) and/or a wired network (e.g., Firewire,Ethernet, etc.). A WAN may be a wired and/or wireless WAN. For example,a WAN may be a personal home or business's wireless network and a WAN isthe Internet, cellular telephone infrastructure, and/or satellitecommunication infrastructure.

FIG. 2 is a schematic block diagram of an embodiment of a computingdevice 12-14. The computing device 12-14 includes a screen-to-screen(STS) communication unit 30, a core control module 40, one or moreprocessing modules 42, one or more main memories 44, cache memory 46, avideo graphics processing module 48, an input/output (I/O) peripheralcontrol module 50, one or more input/output (I/O) interfaces 52, one ormore network interface modules 54, one or more network cards 56-58, oneor more memory interface modules 62 and one or more memories 64-66. Aprocessing module 42 is described in greater detail at the end of thedetailed description of the invention section and, in an alternativeembodiment, has a direction connection to the main memory(s) 44. In analternate embodiment, the core control module 40 and the I/O and/orperipheral control module 50 are one module, such as a chipset, a quickpath interconnect (QPI), and/or an ultra-path interconnect (UPI).

The STS communication unit 30 includes a display 32 with a touch screensensor array 34, a plurality of drive-sense modules (DSM), and a touchscreen processing module 36. In general, the sensors (e.g., electrodes,capacitor sensing cells, capacitor sensors, inductive sensors, etc.) ofthe touch screen sensor array 34 detect a proximal touch of the screen.For example, when one or more fingers touches (e.g., direct contact orvery close (e.g., a few millimeters to a centimeter)) the screen,capacitance of sensors proximal to the touch(es) are affected (e.g.,impedance changes). The drive-sense modules (DSM) coupled to theaffected sensors detect the change and provide a representation of thechange to the touch screen processing module 36, which may be a separateprocessing module or integrated into the processing module 42.

The touch screen processing module 36 processes the representativesignals from the drive-sense modules (DSM) to determine the location ofthe touch(es). This information is inputted to the processing module 42for processing as an input. For example, a touch represents a selectionof a button on screen, a scroll function, a zoom in-out function, anunlock function, a signature function, etc. In an example, a DSMincludes a drive sense circuit (DSC) and a signal source. In a furtherexample, one signal source is utilized for more than one DSM. The DSMallows for communication with a better signal to noise ratio (SNR)(e.g., >100 dB) due at least in part to the low voltage required todrive the DSM. The drive sense module is discussed in greater detailwith reference to one or more subsequent figures.

Each of the main memories 44 includes one or more Random Access Memory(RAM) integrated circuits, or chips. For example, a main memory 44includes four DDR4 (4^(th) generation of double data rate) RAM chips,each running at a rate of 2,400 MHz. In general, the main memory 44stores data and operational instructions most relevant for theprocessing module 42. For example, the core control module 40coordinates the transfer of data and/or operational instructions fromthe main memory 44 and the memory 64-66. The data and/or operationalinstructions retrieved from memory 64-66 are the data and/or operationalinstructions requested by the processing module or will most likely beneeded by the processing module. When the processing module is done withthe data and/or operational instructions in main memory, the corecontrol module 40 coordinates sending updated data to the memory 64-66for storage.

The memory 64-66 includes one or more hard drives, one or more solidstate memory chips, and/or one or more other large capacity storagedevices that, in comparison to cache memory and main memory devices,is/are relatively inexpensive with respect to cost per amount of datastored. The memory 64-66 is coupled to the core control module 40 viathe I/O and/or peripheral control module 50 and via one or more memoryinterface modules 62. In an embodiment, the I/O and/or peripheralcontrol module 50 includes one or more Peripheral Component Interface(PCI) buses to which peripheral components connect to the core controlmodule 40. A memory interface module 62 includes a software driver and ahardware connector for coupling a memory device to the I/O and/orperipheral control module 50. For example, a memory interface module 62is in accordance with a Serial Advanced Technology Attachment (SATA)port.

The core control module 40 coordinates data communications between theprocessing module(s) 42 and the network(s) 15 via the I/O and/orperipheral control module 50, the network interface module(s) 54, andnetwork cards 56 and/or 58. A network card 56-58 includes a wirelesscommunication unit or a wired communication unit. A wirelesscommunication unit includes a wireless local area network (WLAN)communication device, a cellular communication device, a Bluetoothdevice, and/or a ZigBee communication device. A wired communication unitincludes a Gigabit LAN connection, a Firewire connection, and/or aproprietary computer wired connection. A network interface module 54includes a software driver and a hardware connector for coupling thenetwork card to the I/O and/or peripheral control module 50. Forexample, the network interface module 54 is in accordance with one ormore versions of IEEE 802.11, cellular telephone protocols, 10/100/1000Gigabit LAN protocols, etc.

The core control module 40 coordinates data communications between theprocessing module(s) 42 and the STS communication unit 30 via the videographics processing module 48, and the I/O interface module(s) 52 andthe I/O and/or peripheral control module 50. In an embodiment, the STScommunication unit 30 includes or is connected (e.g., operably coupled)to a keypad, a keyboard, control switches, a touchpad, a microphone, acamera, speaker, etc. An I/O interface 52 includes a software driver anda hardware connector for coupling the STS communications unit 30 to theI/O and/or peripheral control module 50. In an embodiment, aninput/output interface 52 is in accordance with one or more UniversalSerial Bus (USB) protocols. In another embodiment, input/outputinterface 52 is in accordance with one or more audio codec protocols.

The processing module 42 communicates with a video graphics processingmodule 48 to display data on the display 32. The display 32 includes anLED (light emitting diode) display, an LCD (liquid crystal display),and/or other type of display technology. The display 32 has aresolution, an aspect ratio, and other features that affect the qualityof the display. The video graphics processing module 48 receives datafrom the processing module 42, processes the data to produce rendereddata in accordance with the characteristics of the display, and providesthe rendered data to the display 32.

FIG. 3 is a schematic block diagram of another embodiment of a computingdevice 12-14 that includes a screen-to-screen (STS) communication unit30, a core control module 40, one or more processing modules 42, one ormore main memories 44, cache memory 46, a video graphics processingmodule 48, one or more input/output (I/O) peripheral control modules 50,one or more input/output (I/O) interface modules 52, one or more networkinterface modules 54, one or more memory interface modules 62, networkcards 56-58 and memories 64-66. The STS communication unit 30 includes adisplay 32 with touch screen sensor array 34 and actuator drive array38, a touch screen processing module 36, a tactile screen processingmodule 39, and a plurality of drive-sense modules (DSM).

Computing device 12-14 operates similarly to computing device 12-14 ofFIG. 2 with the addition of a tactile aspect to the screen 20 as anoutput device. The tactile portion of the display 32 includes aplurality of actuators (e.g., piezoelectric transducers to createvibrations, solenoids to create movement, etc.) to provide a tactilefeel to the display 32. To do so, the processing module creates tactiledata, which is provided to the appropriate drive-sense modules (DSM) viathe tactile screen processing module 39 which may be a stand-aloneprocessing module or integrated into processing module 42. Thedrive-sense modules (DSM) convert the tactile data into drive-actuatesignals and provide them to the appropriate actuators to create thedesired tactile feel on the display 32. In an example, the actuatorsalso may encode data into a vibration to produce a vibration encodeddata signal. For example, a binary 1 is represented as a first vibrationfrequency and a binary 0 is represented as a second vibration frequency.In an instance, the vibration data encoded signal is transmitted toanother computing device via a screen to screen (STS) connection.

A sensor 34 functions to convert a physical input into an electricaloutput and/or an optical output. The physical input of a sensor may beone of a variety of physical input conditions. For example, the physicalcondition includes one or more of, but is not limited to, acoustic waves(e.g., amplitude, phase, polarization, spectrum, and/or wave velocity);a biological and/or chemical condition (e.g., fluid concentration,level, composition, etc.); an electric condition (e.g., charge, voltage,current, conductivity, permittivity, eclectic field, which includesamplitude, phase, and/or polarization); a magnetic condition (e.g.,flux, permeability, magnetic field, which amplitude, phase, and/orpolarization); an optical condition (e.g., refractive index,reflectivity, absorption, etc.); a thermal condition (e.g., temperature,flux, specific heat, thermal conductivity, etc.); and a mechanicalcondition (e.g., position, velocity, acceleration, force, strain,stress, pressure, torque, vibration, etc.). For example, piezoelectricsensor converts force or pressure into an eclectic signal. As anotherexample, a microphone converts audible acoustic waves into electricalsignals.

There are a variety of types of sensors to sense the various types ofphysical conditions. Sensor types include, but are not limited to,capacitor sensors, inductive sensors, accelerometers, piezoelectricsensors, light sensors, magnetic field sensors, ultrasonic sensors,temperature sensors, infrared (IR) sensors, touch sensors, proximitysensors, pressure sensors, level sensors, smoke sensors, and gassensors. In many ways, sensors function as the interface between thephysical world and the digital world by converting real world conditionsinto digital signals that are then processed by computing devices for avast number of applications including, but not limited to, medicalapplications, production automation applications, home environmentcontrol, public safety, and so on.

The various types of sensors have a variety of sensor characteristicsthat are factors in providing power to the sensors, receiving signalsfrom the sensors, and/or interpreting the signals from the sensors. Thesensor characteristics include resistance, reactance, powerrequirements, sensitivity, range, stability, repeatability, linearity,error, response time, and/or frequency response. For example, theresistance, reactance, and/or power requirements are factors indetermining drive circuit requirements. As another example, sensitivity,stability, and/or linearity are factors for interpreting the measure ofthe physical condition based on the received electrical and/or opticalsignal (e.g., measure of temperature, pressure, etc.).

An actuator 38 converts an electrical input into a physical output. Thephysical output of an actuator may be one of a variety of physicaloutput conditions. For example, the physical output condition includesone or more of, but is not limited to, acoustic waves (e.g., amplitude,phase, polarization, spectrum, and/or wave velocity); a magneticcondition (e.g., flux, permeability, magnetic field, which amplitude,phase, and/or polarization); a thermal condition (e.g., temperature,flux, specific heat, thermal conductivity, etc.); and a mechanicalcondition (e.g., position, velocity, acceleration, force, strain,stress, pressure, torque, etc.). As an example, a piezoelectric actuatorconverts voltage into force or pressure. As another example, a speakerconverts electrical signals into audible acoustic waves.

An actuator 38 may be one of a variety of actuators. For example, anactuator is one of a comb drive, a digital micro-mirror device, anelectric motor, an electroactive polymer, a hydraulic cylinder, apiezoelectric actuator, a pneumatic actuator, a screw jack, aservomechanism, a solenoid, a stepper motor, a shape-memory allow, athermal bimorph, and a hydraulic actuator.

The various types of actuators have a variety of actuatorscharacteristics that are factors in providing power to the actuator andsending signals to the actuators for desired performance. The actuatorcharacteristics include resistance, reactance, power requirements,sensitivity, range, stability, repeatability, linearity, error, responsetime, and/or frequency response. For example, the resistance, reactance,and power requirements are factors in determining drive circuitrequirements. As another example, sensitivity, stability, and/or linearare factors for generating the signaling to send to the actuator toobtain the desired physical output condition.

As a specific example of operation, the actuators 38 generate avibration encoded signal based on digital data as part of a screen toscreen (STS) communication with another computing device 12-14. Thevibration encoded signal vibrates through and/or across a transmissionmedium (e.g., surface (e.g., of table, of a body, etc.) from a computing12-14 to another computing device 12-14. The other computing device12-14 receives the vibration encoded signal via its sensors 34 (e.g.,transducers) and decodes the vibration encoded data signal to recoverthe digital data.

FIG. 4 is a schematic block diagram of another embodiment of a computingdevice 12-14 that includes a screen-to-screen (STS) communication unit30, a core control module 40, one or more processing modules 42, one ormore main memories 44, cache memory 46, one or more input/output (I/O)peripheral control modules 50, an output interface module 53, an inputinterface module 55, one or more network interface modules 54, and oneor more memory interface modules 62, network cards 56-58 and memories64-66. In this embodiment, the STS communication unit 30 includes a minidisplay 59, a touch screen processing module 36, a touch screen withsensors 57, and a plurality of drive sense modules.

FIG. 5 is a schematic block diagram of another embodiment of a computingdevice 12-14 that includes a screen-to-screen (STS) communication unit30, a core control module 40, one or more processing modules 42, one ormore main memories 44, cache memory 46, one or more input/output (I/O)peripheral control modules 50, an output interface module 53, an inputinterface module 55, one or more memory interface modules 62, and memory64. The STS communication unit 30 includes mini display 59, a touchscreen with sensors 57, a touch screen processing module 36 and aplurality of drive sense modules (DSM).

FIG. 6 is a schematic block diagram of another embodiment of a computingdevice 12-14 that includes a screen-to-screen (STS) communication unit30, a core control module 40, one or more processing modules 42, one ormore main memories 44, cache memory 46, a video graphics processingmodule 48, one or more input/output (I/O) peripheral control modules 50,one or more input/output (I/O) interface modules 52, one or more networkinterface modules 54, one or more memory interface modules 62, networkcards 56-58 and memories 64-66.

In this embodiment, the STS communication unit has a display 32 withtouch screen sensor array 34 and a separate touch screen sensor array34-1. Each of the display 32 with touch screen sensor array 34 and touchscreen sensor array 34-1 are connected to a touch screen processingmodule 36 via a plurality of drive sense modules (DSM). In a specificexample, the touch screen sensor array 34-1 is a single electrode orsensor (e.g., button, control point, etc.).

There is a variety of locations for which to locate the display 32 andthe touch screen senor array 34-1 on the computing device 12-14. Someexample include, but are not limited to the following. In a firstexample, the display 32 with touch screen sensor array 34 is located ona front the computing device and the touch screen with sensor array 34-1is located on a side of the computing device. In a second example, thedisplay 32 with touch screen sensor array 34 is located on a front thecomputing device and the touch screen with sensor array 34-1 is locatedon a back of the computing device. In a third example, the display 32with touch screen sensor array 34 is located on a front and/or side thecomputing device and the touch screen with sensor array 34-1 is locatedon a front of the computing device.

FIG. 7 is a schematic block diagram of another embodiment of a computingdevice 12-14 that includes a screen-to-screen (STS) communication unit30, a core control module 40, one or more processing modules 42, one ormore main memories 44, cache memory 46, a video graphics processingmodule 48, one or more input/output (I/O) peripheral control modules 50,one or more input/output (I/O) interface modules 52, one or more networkinterface modules 54, one or more memory interface modules 62, networkcards 56-58 and memories 64-66.

In this embodiment, the STS communication unit 30 has a display 32 withtouch screen sensor array 34 and touch sensor 70. The display 32 withtouch screen sensor array 34 and touch screen sensor array 34 areconnected to a touch screen processing module 36 via a plurality ofdrive sense modules (DSM) and the touch sensor 70 is also operablecoupled to the touch screen processing module 36 via another DSM.

In an example, the touch sensor 70 is a single electrode. In anotherexample, the touch sensor is a capacitive sensor. The touch sensor 70may be on the front of computing device 12-14, may be on the back ofcomputing device 12-14 and/or may be on one or more sides of thecomputing device 12-14. As a specific embodiment, the computing deviceis a cell phone with a display on the front and the touch sensor on theside.

FIG. 8 is a schematic block diagram of another embodiment of a computingdevice 12-14 that includes a screen-to-screen (STS) communication unit30, a core control module 40, one or more processing modules 42, one ormore main memories 44, cache memory 46, one or more input/output (I/O)peripheral control modules 50, an output interface 53, an inputinterface 55, one or more memory interface modules 62, and a memory 64.The STS communication unit 30 includes a touch screen processing module82, a drive sense module 100 (e.g., the drive sense module (DSM) of FIG.1), and a touch sensor 70. In an example, the touch sensor 70 is asingle electrode. In another example, the touch sensor is a capacitivesensor.

FIG. 9 is a schematic block diagram of an embodiment of a touch screendisplay with sensors 55 that includes a plurality of drive-sense modules(DSM), a touch screen processing module 36, a display 83, and a touchscreen sensor array 34. The touch screen display with sensors 55 iscoupled to a processing module 42, a video graphics processing module48, and memory 64 and/or 66, which are components of a computing device(e.g., 12-14), an interactive display, or other device that includes atouch screen display. An interactive display functions to provide userswith an interactive experience (e.g., touch the screen to obtaininformation, to input information, to be entertained, to complete atransaction, etc.). For example, a store provides interactive displaysfor customers to order products, to find certain products, to obtaincoupons, to enter contests, to sign up for store rewards, to learninformation associated with a product, and many other functions.

There are a variety of other devices that include a touch screendisplay. For example, a vending machine includes a touch screen displayto select and/or pay for an item. As another example of a device havinga touch screen display is an Automated Teller Machine (ATM). As yetanother example, an automobile includes a touch screen display forentertainment media control, navigation, climate control, vehicleinformation (e.g., tire air pressure, gas levels, etc.), etc. As a stillfurther example, a smart device (e.g., light switch, home securitycontrol hub, thermostat, etc.) within a home includes a touch screen.

In an example, the touch screen display with sensors 55 includes a largedisplay 83 that has a resolution equal to or greater than fullhigh-definition (HD), an aspect ratio of a set of aspect ratios, and ascreen size equal to or greater than thirty-two inches. The followingtable lists various combinations of resolution, aspect ratio, and screensize for the display 83, but it is not an exhaustive list.

Width Height pixel screen screen Resolution (lines) (lines) aspect ratioaspect ratio size (inches) HD (high 1280 720 1:1 16:9 32, 40, 43, 50,55, 60, 65, definition) 70, 75, &/or >80 Full HD 1920 1080 1:1 16:9 32,40, 43, 50, 55, 60, 65, 70, 75, &/or >80 HD 960 720 4:3 16:9 32, 40, 43,50, 55, 60, 65, 70, 75, &/or >80 HD 1440 1080 4:3 16:9 32, 40, 43, 50,55, 60, 65, 70, 75, &/or >80 HD 1280 1080 3:2 16:9 32, 40, 43, 50, 55,60, 65, 70, 75, &/or >80 QHD (quad 2560 1440 1:1 16:9 32, 40, 43, 50,55, 60, 65, HD) 70, 75, &/or >80 UHD (Ultra 3840 2160 1:1 16:9 32, 40,43, 50, 55, 60, 65, HD) or 4K 70, 75, &/or >80 8K 7680 4320 1:1 16:9 32,40, 43, 50, 55, 60, 65, 70, 75, &/or >80 HD and 1280->=7680 720->=43201:1, 2:3,  2:3 50, 55, 60, 65, 70, above etc. 75, &/or >80

The display 83 is one of a variety of types of displays that is operableto render frames of data into visible images. For example, the displayis one or more of: a light emitting diode (LED) display, anelectroluminescent display (ELD), a plasma display panel (PDP), a liquidcrystal display (LCD), an LCD high performance addressing (HPA) display,an LCD thin film transistor (TFT) display, an organic light emittingdiode (OLED) display, a digital light processing (DLP) display, asurface conductive electron emitter (SED) display, a field emissiondisplay (FED), a laser TV display, a carbon nanotubes display, a quantumdot display, an interferometric modulator display (IMOD), and a digitalmicroshutter display (DMS). The display is active in a full display modeor a multiplexed display mode (i.e., only part of the display is activeat a time).

The display 83 further includes touch screen sensor array 34 thatprovide the sensors 34 for the touch sense part of the touch screendisplay. The sensor array 34 is distributed throughout the display areaor where touch screen functionality is desired. For example, a firstgroup of sensors of the sensor array 34 are arranged in rows and asecond group of sensors of the sensor array 34 are arranged in columns.Note the row sensors may be separated from the column sensors by adielectric material.

The sensor array 34 is comprised of a transparent conductive materialand are in-cell or on-cell with respect to layers of the display. Forexample, a conductive trace is placed in-cell or on-cell of a layer ofthe touch screen display. The transparent conductive material, which issubstantially transparent and has negligible effect on video quality ofthe display with respect to the human eye. For instance, a sensor of thesensor array 34 is an electrode and is constructed from one or more of:Indium Tin Oxide, Graphene, Carbon Nanotubes, Thin Metal Films, SilverNanowires Hybrid Materials, Aluminum-doped Zinc Oxide (AZO), AmorphousIndium-Zinc Oxide, Gallium-doped Zinc Oxide (GZO), and poly polystyrenesulfonate (PEDOT).

In an example, the sensors are electrodes. As such, the rows ofelectrodes intersecting with the column of electrodes form a capacitivegrid. For each intersection of a row and column electrode, a mutualcapacitance (Cm) exists. In addition, each electrode (row and column)has a self-capacitance (Cs) with respect to a ground reference of thetouch screen. As such, the touch screen senor array includes a pluralityof mutual capacitances (Cm) and a plurality of self-capacitances (Cs),where the number of mutual capacitances equals the number of rowsmultiplied by the number of columns and the number self-capacitancesequals the number of rows plus the number of columns.

In general, changes to the self and/or mutual capacitances result fromchanges in the dielectric properties of the capacitances. For example,when a human touches the touch screen, self-capacitance increases andmutual capacitance decreases due the dielectric properties of the personand the coupling of the person to the ground reference of the computingdevice. In another example, when an object is placed on the touch screenwithout a connection to ground, the mutual capacitances will increase ordecrease depending on the dielectric properties of the object. Thisallows for different types of objects to be identified (e.g., touchscreen pen, finger, another computing device proximal to touch screenfor setting up an STS connection, etc.).

The memory 64 and/or 66 store an operating system 89, a screen-to-screen(STS) communication application 90, one or more STS source userapplications 91 and one or more payment applications 92. The STScommunication application 90 functions to allow STS communications fromone computing device to another. For example, the STS communicationapplication 90 works with an STS communication application on the otherdevice to establish an STS communication protocol for the STS wirelessconnection 18. As a further example, the STS communication applicationstores and/or has access to verify personal data (e.g., biometric data,password, etc.) of an authorized user of the device before enabling theSTS communication.

The source user applications 91 includes, but are not limited to, avideo playback application, a spreadsheet application, a word processingapplication, a computer aided drawing application, a photo displayapplication, an image processing application, a database application,and a plurality of interactive user applications, etc. While executing asource user application 91, the processing module generates data fordisplay (e.g., video data, image data, text data, etc.). The paymentapplications 92 includes, but are not limited to, a bank application, apeer-to-peer payment application, a credit card payment application, adebit card payment application, a gift card payment application, etc.Note the STS communication applications 90 and source user applications91 are OS agnostic (e.g., are operable to function on a variety ofoperating systems (e.g., Mac OS, Window OS, Linux OS, etc.)).

In an example of operation of an STS communication, the touch screenprocessing module 36 sends display data to the video graphics processingmodule 48, which converts the data into frames of video 87. The videographics processing module 48 sends the frames of video 87 (e.g., framesof a video file, refresh rate for a word processing document, a seriesof images, etc.) to the display interface 93. The display interface 93provides the frames of video to the display 83, which renders the framesof video into visible images.

While the display 83 is rendering the frames of video into visibleimages, the drive-sense modules (DSM) provide outbound signals of theSTS communication to the sensors of the touch screen sensor array 34 andreceive inbound signals of the STS communication from the sensors. Whenthe screen is proximal to another screen or receiving signals via bodyas a network (BaaN), capacitance of the sensors are changed by thesignals from the other screen. The DSMs detect the capacitance changefor affected sensors and provide the detected change to the touch screenprocessing module 36.

The touch screen processing module 36 processes the capacitance changeof the affected sensors to determine one or more specific elements(e.g., bit, byte, data word, symbol, etc.) of the STS communication andprovides this information to the processing module 36. Processing module36 processes the one or more specific elements to determine a portion ofthe STS communication. For example, the specific element indicates oneor more of a purchase, a quantity, an edit, an identity of an item, apurchase price, a digital signature, a security code, and anacknowledgement.

FIG. 10 is a schematic block diagram of another embodiment of a touchscreen sensor array 34 that includes a plurality of drive-sense modules(DSM), the processing module 42, and memory 64 and/or 66. The touchscreen display operates similarly to the touch screen display 55 of FIG.9 without the display 83, display interface 93 and video graphicsprocessing module 48.

FIG. 11 is a schematic block diagram of an embodiment of a drive sensemodule (DSM) 100 connected to an electrode 105. The DSM 100 includes asignal source circuit 102 and a drive sense circuit (DSC) 103. Thesignal source 102 includes an alternate current (AC) signal generator,an existing element of computing device 12-14, display data that isemanated from a display, and/or another signal source.

The DSC 103 includes an analog front end 104, an analog to digitalconverter (ADC) & digital to analog converter (DAC) 106, and a digitalprocessing circuit 108. The analog front end includes one or moreamplifiers, filters, mixers, oscillators, converters, voltage sources,current sources, etc. For example, the analog front end 104 includes acurrent source, an ADC, a DAC and a comparator.

The analog to digital converter (ADC) 106 may be implemented in avariety of ways. For example, the (ADC) 106 is one of: a flash ADC, asuccessive approximation ADC, a ramp-compare ADC, a Wilkinson ADC, anintegrating ADC, a delta encoded ADC, and/or a sigma-delta ADC. Thedigital to analog converter (DAC) 106 be implemented in a variety ofways. For example, the DAC 106 is one of: a sigma-delta DAC, a pulsewidth modulator DAC, a binary weighted DAC, a successive approximationDAC, and/or a thermometer-coded DAC. The digital processing circuit 108includes one or more of digital filtering (e.g., decimation and/orbandpass filtering), format shifting, buffering, etc. Note in anembodiment, the digital processing circuit includes the ADC DAC 106.

In an example of operation, the DSM produces a digital inbound signal107 that is representative of changes to an electrical characteristic(e.g., an impedance, a current, a reactance, a voltage, a frequencyresponse, etc.) of the electrode 105 due to an STS communication. Inparticular, the analog front end 104 receives an analog reference signal101 from the signal source 102 and utilizes it to determine the changein the electrical characteristic of the electrode. The analog front end104 outputs a representation of the change to the ADC DAC 106, whichconverts it into a digital signal. The digital processing 108 processesthe digital signal to produce digital inbound signal 107, whichrepresents an element of the STS communication.

To transmit an element of the STS communication, the digital processingconverts digital outbound signal 109 (e.g., representation of theelement) into an analog outbound signal 109-1. The signal source 102generates an analog reference signal 101 based on the analog outboundsignal 109-1. For example, the analog outbound signal 109-1 indicateswhether an analog reference signal is to be generated, and if so, atwhat frequency. As another example, the signal source 102 modulates acarrier signal with the analog outbound signal 109-1 to produce theanalog reference signal 101. The analog front end 104 processes theanalog reference signal to drive an analog signal representing theelement onto electrode 105. Further examples of the operation of thedrive sense circuit (DSC) 103 are discussed in patent pendingapplication Ser. No. 16/113,379, entitled Drive Sense Circuit WithDrive-Sense Line, filed Aug. 27, 2018.

FIG. 12A is an embodiment of a portion (e.g., the analog front end 104,and an ADC 106-1 of the ADC DAC 106) of the drive sense circuit 103. Inthis embodiment, the analog front end 104 includes a current source 111and a comparator 112.

In the example of receiving an element (e.g., bit, byte, data word,symbol, etc.) of an STS communication, the comparator 112 produces ananalog compensation signal/analog feedback signal based on comparing ananalog reference signal 101 to signaling 110, which is indicative of anelectrical characteristic (e.g., impedance (Z)) change to electrode 105.The ADC 106-1 converts the analog compensation signal to produce adigital inbound signal that represents the element of the STScommunication. The dependent current source 111 modifies a current (I)on the output line (e.g., connected to electrode 105) based on theanalog feedback signal so that a voltage (V) on the electrode remainssubstantially constant. For example, when an impedance (Z) decreases onelectrode 105, according to the formula V=I*Z, the current is increasedsuch that the voltage on the electrode remains substantially constant.

FIG. 12B is a schematic block diagram of another embodiment of a portionthe drive sense circuit 103 that includes a, ADC 106-1, a DAC 106-2 acurrent source 111, and a comparator 112. This example is similar toFIG. 12A, except the feedback loop to the current source 111 is throughthe ADC 106-1 and the DAC 106-2, instead of directly from the comparator112.

FIG. 13 is a schematic block diagram of a plurality of drive sensemodules 100. The drive sense modules are configured similar to FIG. 12,except that the digital processing 108 includes an analog to digitalconverter and a digital to analog converter and one signal source 102provides the analog reference signal 101 to more than one drive sensemodule (DSM) 100. Further, analog outbound signal 114 is sent directlyto analog front end 104. The analog front end 104 also provides analogyinbound signal 116 to digital processing 108.

FIG. 14 is a schematic block diagram of an example of a user computingdevice (UCD) 14 communicating with an interactive computing device (ICD)12 via a screen-to-screen (STS) wireless connection 18. The usercomputing device (UCD) 14 may be implemented by a combination of two ormore devices. For example, the user computing device 14 is a cell phoneand a fob (e.g., small security hardware device with built-inauthentication (e.g., keyless entry device, remote car starter, garagedoor opener, etc.)). As another example, the user computing device 14 isa cell phone and a car. Alternatively, the user computing device 14 isan individual device such as a cell phone, a tablet, a personal touchscreen device (e.g., fob), a car, etc. The user computing device 14includes a computing core 40 connected to a user input interface 144, auser output interface 146, an STS communication unit 30, and a memory 64and/or 66.

The memory 64 and/or 66 of UCD 14 includes an operating system 89, anSTS communication application 90, a set (e.g., one or more) of userinteraction applications 148, a set of payment applications 92, andconfidential information 141. The confidential information 141 includes,but is not limited to, user's personal information, user computingdevice identification (ID), user's payment information, securityinformation (e.g., passwords, biometric data, etc.) and user's personalpreferences per user application (e.g., preference for coffee orders,fast food orders, transportation tickets, event tickets, etc.).

As some limited examples, the set of user interaction applications 148includes a fast food drive ordering application, a transportation ticketpurchase application, an event ticket purchase application, a bankingapplication, a point of sale payment application, a rental car enableand checkout application, an airline application, a sales informationapplication, an interactive screen information application, a datatransfer application, a meeting data exchange application, a hotel checkin application, and a cell phone is hotel room key application.

The STS communication application 90 functions as previously describedto assist the UCD 14 in setting up the communication between devices.For example, the STS communication application 90 determines (e.g.,selects a default, receives a command, etc.) one or more of acommunication medium (e.g., close proximity, body as a network, surface,etc.), a communication method (e.g., cellular data, STS communicationlink, Bluetooth, etc.), a signaling and/or pattern protocol (e.g.,amplitude shift keying (ASK) modulation, etc.), and security mechanisms(e.g., security codes, encryption, data transmission of particular datatypes restrictions, etc.) for which the devices utilize for thecommunication. The payment applications 92 include, but are not limitedto, one or more of a bank application, a credit card application,peer-to-peer payment application, and a cryptocurrency exchangeapplication.

The interactive computing device (ICD) 12 includes a screen to screen(STS) communication unit 30, a computing core 40, and a memory 64 and/or66. The memory 64 and/or 66 of the ICD 12 includes an STS communicationapplication 90, an operator interaction application 140, a set ofpayment processing applications 142, and confidential information 141.

The STS communication application 90 of the ICD 12 functions similarlyas the STS communication application 90 of the UCD 14 to setup the STScommunications from an operator of the ICD's perspective. As an examplein setting up communication between the devices, the STS communicationapplication of the ICD is a leader (controls communication settings) andthe STS communication application of the UCD is a follower (e.g., usessettings selected by the ICD STS communication app 90). In anotherexample, the STS communication application 90 of both the UCD 14 and theICD 12 need to agree on and/or have control over various settings. Forexample, the UCD 14 and ICD 12 agree to use a cellular data connection(e.g., 5G) to transmit transactional data. However, the UCD will onlytransmit certain confidential information via an STS wireless connection18 and the ICD will only accept connections with a minimum bit rate overa wireless local area network (WLAN) connection with the UCD. Thus, theICD needs to agree to receive the certain confidential information viathe STS wireless connection 18 and the UCD needs to agree to transmit atthe minimum bit rate over the WLAN to successfully perform the setup.

The operator interaction application 140 includes an operator version ofa fast food drive ordering application, a transportation ticket purchaseapplication, an event ticket purchase application, a bankingapplication, a point of sale payment application, a rental car enableand checkout application, an airline application, a sales informationapplication, an interactive screen information application, a datatransfer application, a meeting data exchange application, a hotel checkin application, a cell phone is hotel room key application. The paymentprocessing application 142 includes one or more of a bank operatorapplication, a credit card operator application, peer-to-peer paymentoperator application, a cryptocurrency exchange operator application,and an automated clearing house application.

Once the STS communication settings are agreed upon, the UCD 14 and ICD12 may utilize the STS wireless connection 18 to transmit data of atransaction. The STS wireless connection 18 includes one or moreconnection types. For example, a first connection type is a body as anetwork (BaaN) connection. As another example, a second connection typeis a touch screen to touch screen close proximity connection. As yetanother example, a third connection type is a connective surface betweenthe touch screen to touch screen (e.g., in order to transmit an encodedvibration signal). In an example, the user computing device 14 and theinteractive computing device 12 exchange confidential information (e.g.,confidential information 141), or a portion thereof via the STS wirelessconnection 18.

By using the STS wireless connection, the UCD 14 and ICD 12 exchangedata in a secure manner and also reduce the amount of steps a user ofthe UCD needs to manually complete to perform a transaction. Forexample, using a BaaN connection, the signal is difficult for any deviceother than then UCD and ICD to detect. Further, when transmittingpayment information during touching a screen to confirm an order ofitems, a user does not have to perform one or more of the steps oflocating a credit card, swiping the card, verifying the amount, signinga screen or physical receipt, and returning card to a safe location.

FIG. 15 is a schematic block diagram of an embodiment of ascreen-to-screen (STS) connection 18 between a user computing device(UCD) 14 and an interactive computing device (ICD) 12 through a body 232(e.g., human body for a body as a network (BaaN) STS connection). TheUCD 14 and ICD 12 include a touch screen sensor array 34, drive sensemodules, and a touch screen processing module 36. The touch screensensor array 34 includes rows of electrodes 105 (shown in yellow) andcolumns of electrodes 105 (shown in blue).

In an example of operation, a drive sense module generates a signalhaving an oscillation component based on a command from the touch screenprocessing module 36. The drive sense module drives the signal onto atouch sense element (e.g., one or more electrodes 105) of the touchscreen sensor array 34. When a part of the body (e.g., finger, hand,arm, foot, etc.) touches the first touch sense element or is in closeproximity (e.g., within a few millimeters to tens of millimeters), thesignal on the touch sense element propagates through the body 232. TheICD 12 receives the signal through another part of the body 232 (e.g.,another finger) via a second touch (or close proximity connection) onthe touch screen sensor array 34 of the ICD 12.

As such, data is securely transmitted from one device to another. Thetransmit of data is also more efficient for a user (e.g., body 232) asthe data can be transmitted more seamlessly than other communicationtypes. For example, with STS communications enabled on both the UCD andthe ICD, when a user of the UCD presses (e.g. touches) a payment buttonon the ICD, payment information may be security transmitted from the UCDto the ICD via the STS connection 18 during the pressing without othersteps (e.g., inputting payment information, selecting a payment option,scanning a bar code, swiping a card, etc.).

In a specific embodiment, the touch screen processing module may adjustthe current of a signal driven onto the touch sense element based on acomposition of the body in the BaaN. For example, a user's bodyimpedance lowers as total body water of the user (e.g., stored in theuser's tissues) increases. Thus, as the users' impedance changes, thetouch screen processing module may adjust the current accordingly. Thisallows the current usage to be minimized, which may save power. Thisfurther allows for the signal to be modified to achieve desired signalcharacteristics (e.g., signal to noise ratio, signal strength, etc.).

FIG. 16 is a schematic block diagram of an embodiment of ascreen-to-screen (STS) connection 18 between a user computing device(UCD) 14 and an interactive computing device (ICD) 12 through a humanbody 232. The UCD 14 includes an electrode 105 (e.g., of a touchscreen), a drive sense module, and a touch screen processing module 36.The ICD 12 includes a touch screen sensor array 34 that includeselectrodes 105, drive sense modules (DSMs), and a touch screenprocessing module 36.

In an example of operation, the STS connection 18 is formed between anelectrode 105 of the UCD 14 and a touch sense element (e.g., one or moreelectrodes) of the touch screen sensor array 34 of the ICD 12. The DSMssense an impedance change of a corresponding electrode(s) 105, which isinterpreted by a touch screen processing module 36 as a command. As aspecific example, the command is a user signature. While the user issigning an area of the touch screen sensor array 34, an STS connection18 is formed and data (e.g., payment data) can be exchanged between theUCD and the ICD over the STS connection. Thus, during the signature,data transmitted via the STS connection 18 assist in completing atransaction.

FIG. 17 is a schematic block diagram of an example of a screen-to-screen(STS) connection 18 between a user computing device (UCD) 14 and aninteractive computing device (ICD) 12 through multiple bodies 232. TheUCD 14 includes a first touch screen sensor array 34 and the ICD 12includes a second touch screen sensor array 34.

In an example of operation, the STS connection 18 is formed from thefirst touch sensor array 34 through a first body 232 and a second body232 to a second touch screen sensor array 34 of ICD 12, or vice versa.There are various ways a connection between the bodies can occur. Forexample, the connection occurs when user 1 and user 2 first bump, shakehands or otherwise have skin-to-skin contact that allows the signal(e.g., driven onto a touch sense element of the touch screen) topropagate. In a specific example, the STS connection 18 is formedbetween the UCD 14 and the ICD 12 when the body #1 232 is in contactwith the body #2 232 for a certain time period (e.g., 20 milliseconds,0.2 seconds, 3 seconds, etc.).

In an embodiment, the computing device 12-14 includes a touch button orother specific area on the computing device 12-14 used to ensurepurposeful engagement of a user in sharing data via the STS connection18. For example, a portion of a side of the computing device is selected(e.g., clicked, swiped, etc.) 3 times as a command to purposefullyengage. As another example, a portion of a display on the computingdevice 12-14 displays a share “button” for a user to select in order topurposefully engage. As yet another example, “shaking” the computingdevice 12-14 indicates the user's intent to purposefully engage.

FIG. 18 is a schematic block diagram of an example of forming multiplescreen to screen (STS) connections 18 for a transaction between aninteractive computing device (ICD) 12 and multiple user computingdevices (UCDs) 14. In an example, a first STS connection 18 is formedbetween the UCD #1 and the ICD 12 via a first body 232 and a second STSconnection 18 is formed between the ICD 12 and the UCD #2 via a secondbody 232. In an example, this allows the UCD #1 and the UCD #2 to sharedata via the first and second STS connections 18 and the ICD 12.

As a specific example, multiple users determine to split a dinner billat restaurant. For example, a user #1 of UCD #1 (a first cell phoneoperable to perform STS communications) and user #2 of UCD #2 (a secondcell phone operable to perform STS communications) determine to splitthe dinner bill. The ICD 12, which is a point-of-sale device thatincludes a touch screen sensor array 34 and is operable to perform STScommunications. User #1 and #2 both activate a payment transaction via apayment application on their cell phone and touch the touch screensensor array 34 of the point of sale device, which forms an STSconnection 18 from each cell phone to the point of sale device.

The point of sale device prompts the users to select items for whichthey will provide payment or prompts the users to select a percentage ofthe bill they will pay. For example, user #1 indicates they will pay 60%of the bill amount and user #2 indicates they will pay 40% of the billamount. In a specific embodiment, the users must touch the touch screensensor array during the same time period (e.g., simultaneously, within 1sec, etc.) to properly validate the transaction.

FIG. 19 is a schematic block diagram of an example of a screen-to-screen(STS) connection 18 between a user computing device (UCD) 14 and aninteractive computing device (ICD) 12. In this example, another bodypart 233 (e.g., leg, arm, chest, wrist, etc.) in is contact with the UCD14 and operates to transmit a signal between the UCD 14 and the ICD 12via the STS connection 18 that includes the other body part 233, thebody 232, and hand 235. As a specific example, the UCD 14 is a cellphone in a user's pants pocket and the other body part is a leg proximalto the pants pocket, allowing the user to transmit STS communicationswithout removing the cell phone from their pocket.

FIG. 20 is a schematic block diagram of an example of transmitting closeproximity signals 127 from a user computing device 14 to an interactivecomputing device 12 to form a screen to screen (STS) connection 18. Theuser computing device 14 includes drive sense modules (DSMs) and a touchscreen array 34 of electrodes 105. The interactive computing device 14includes drive sense modules (DSMs) and a touch screen array 34 ofelectrodes 105.

In an example of operation, data is transmitted in close proximitysignals 127 via one or more electrodes 105 of the user computing device(UCD) 14 touch screen with an array 34 of electrodes 105. The electrodes105 are shaped and designed for capacitance sensing (e.g., not radiofrequency (RF) transmission). In an example, the electrodes of thecomputing device generate and shape an electric field. At closeproximity (e.g., a few centimeters (cm) to 10's of cm (e.g., 70 cm),electrodes in another computing device will detect the electric field.In this example, the signaling is very low power and the radiated energyfrom the signal drops off very rapidly (e.g., less than few feet beforesignal to noise ratio is too low).

In an example, the UCD 14 selects one or more of the electrodes 105 totransmit the close proximity signals 127. For example, the UCD 14determines an optimal area (e.g., which contains one or more electrodes)of the touch screen sensor array 34 to transmit to produce the selectedelectrodes 105. As another example, the UCD 14 selects electrodes forreceiving close proximity signals 127 to be transmitted from the IDC 12.Note the UCD may select one or more different electrodes for receivingand transmitting the close proximity signals 127.

FIG. 21 is a schematic block diagram of another example of transmittingclose proximity signals 127 between a user computing device (UCD) 14 andan interactive computing device 12 to form a screen to screen (STS)connection 18. The UCD includes a single electrode 105. The ICD 12includes an array of electrodes 105 and is enabled to receive the closeproximity signal 127 on one or more of any of the electrodes 105. Theelectrode 105 of the UCD may be one of a variety of shapes. For example,the electrode shape is one or more of a rectangle, a polygon, circular,a meandering trace, and a square.

FIG. 22 is a schematic block diagram of another example of transmittingclose proximity signals 127 between a user computing device (UCD) 14 andan interactive computing device (ICD) 12 to form a screen to screen(STS) connection. In this example, each touch screen (e.g., of the usercomputing device, of the interactive computing device) includes a singleelectrode 105. The orientation of an electrode of one device can varywith an electrode of another device. For example, the electrodes may beoriented perpendicular to each other, parallel to each other, offset(e.g., from a horizontal center, from a vertical center, etc.) withrespect each other, and/or a rotated a certain number of degrees withrespect to each other. In this example, the electrode 105 (shown inyellow) of the UCD 14 transmits close proximity signals 127 (shown asthe yellow signal) to the electrode 105 (shown in blue) of the ICD 12.The electrode 105 (shown in blue) of the ICD 12 transmits closeproximity signals 127 (shown as the blue signal) to the electrode 105 ofthe UCD 14.

FIG. 23 is a logic flow diagram of an example of a method of a first andsecond computing device (e.g., a user computing device, an interactivecomputing device, another computing device, etc.) communicating via ascreen to screen (STS) connection. The method begins or continues withstep 160, where a first computing device generates a signal. Forexample, the signal has one or more of a direct current (DC) componentand an oscillating component. The method continues with step 162, wherethe first computing device drives the signal on to a first touch senseelement (e.g., one or more electrodes) of the first computing device.

The method continues with step 163, where the first computing devicedetermines whether it detects a touch (e.g., pen, human finger, etc) onthe first touch sense element based on the signal. For example, thefirst computing device detects the touch by determining a capacitancechange (e.g., self-capacitance, mutual-capacitance) associated with thefirst touch sense element. When the touch is not detected, the methodcontinues with back to step 163. Alternatively when the touch is notdetected, the method times out or loops back to steps 160 and/or 162.

When the touch is detected, the method continues at step 164, where thefirst computing device modulates the signal with data to produce amodulated data signal. In an example, the oscillating component of thesignal has a first frequency and the first computing device modulatingthe signal with the data to produce the modulated data signal includesmixing the signal with the data that includes a second oscillatingcomponent having a second frequency.

The method continues with step 166, where the second computing devicereceives the modulated data signal via a transmission medium and asecond touch sense element (e.g., one or more second electrodes) of thesecond computing device. The transmission medium includes at least oneof a human body (e.g., body as a network (BaaN)) and a close proximity(e.g., 70 cm or less) between the first and second computing devices. Inan example, when the human body is the transmission medium, the secondcomputing device operates to detect a second touch on the second touchsense element.

The method continues with step 168, where the second computing devicedemodulates the modulated data signal to recover the data. In anexample, the second computing device may respond to the data bygenerating a second signal having a second oscillating component. Thesecond computing device drives the second signal on the second touchsense element and detects a second touch on the second touch senseelement based on the second signal. While the second touch is detected,the second computing device modulates the second signal with second datato produce a second modulated data signal. For example, the secondcomputing device backscatters the second data with the modulated datasignal to produce the second modulated data signal. As another example,the second computing device mixes the second data with the second signalto include a second oscillating component having a second frequency.

The first computing device may then receive the second modulated datasignal via the transmission medium and the first touch sense elementand/or another touch sense element (e.g., touch sense element in contactwith a user) of the first computing device. The first computing devicedemodulates the second modulated data signal to recover the second data.

FIG. 24 is a schematic block diagram of a computing device 12-14 thatincludes a computing core 40, a screen-to-screen (STS) communicationunit 30, a cellular communication unit 122, a wireless local areanetwork (WLAN) communication unit 124, and a Bluetooth (BT)communication unit 126. As such, a computing device 12-14 cancommunicate in various forms (e.g., via Bluetooth, via STS, etc.) withother devices (e.g., servers, other computing devices, base stations,etc.) via one or more of the communication units 120-126. For example, afirst computing device 12-14 communicates STS data of a transaction toanother computing device 12-14 via the STS communication unit 30. Asanother example, a computing device 12-14 communicates verification dataof a transaction to an interactive application server via the cellularcommunication unit 122.

The computing device determines one or more of the communication options(e.g., screen-to-screen STS, Bluetooth (BT), etc.) to use based on adata type and/or a data communication protocol. For example, the datacommunication protocol indicates to communicate data of a privatepersonal data type via the STS communication unit 30. As anotherexample, the computing device determines to communicate user computingdevice location information via the cellular communication unit 122.Further examples of communicating data via the one or more communicationunits 120-126 is discussed in further detail with reference to one ormore subsequent figures.

FIG. 25 is a schematic block diagram of an embodiment of an example of acommunication that includes a user computing device (UCD) 14, aninteractive computing device (ICD) 12, an interaction application server20, a screen-to-screen (STS) communication server 22, a paymentprocessing server 24, an independent server 26, a local server 132, anaccess point 134, and a cellular data base station 130. The local server132 and the access point 134 may be connected via a wired and/orwireless connection. The user computing device (UCD)14 may be a cellphone and/or a personal device (e.g., a device that stores personal,private, confidential and/or sensitive information regarding a user).

In this example, the user computing device 14, the interactive computingdevice 12, the cellular data base station 130 and the access point 134communicate with each other via one or more particular communicationtypes in accordance with a communication protocol. The communicationtype is based on one or more of the type of device (e.g., ICD, UCD,server, etc.), the communication requirements (e.g., a minimum signal tonoise ratio (SNR), a minimum bit rate, etc.) and the type of data (e.g.,local data, individual data, global data, etc.) being communicated. Forexample, the user computing device 14 and the access point 134communicate local data via a wireless local area network (WLAN)communication. As another example, the user computing device 14 and thecellular data base station 130 communicate global data via a cellulardata communication. As yet another example, the user computing device 14and the interactive computing device 12 communicate individual data viaan STS communication. In an example, individual data is data that ispersonal, private, sensitive and/or otherwise confidential at the timeof the conveyance of the individual data.

By using multiple communication types, data is communicated between thedevices more efficiently and securely. For example, the user computingdevice 14 uses a 5G communication (e.g., fastest connection available)to download global data from the interaction application server 20 anduses an STS communication (e.g., most secure connection available) tosend payment data to the interactive computing device 12. Note the twoor more of the communications may occur concurrently.

FIG. 26 is a schematic block diagram of an example of a communicationthat is similar to FIG. 25, except that user computing device (UCD) 14and interactive computing device (ICD) 12 also communicate the localdata with each other via a Bluetooth (BT) communication. Thus, the UCD14 and the ICD 12 may communicate data via one or more of a wirelesslocal area network (WLAN) communication, a Bluetooth communication, ascreen to screen (STS) communication and a cellular data communication.Note in an example, the various communication paths are utilizedconcurrently.

FIG. 27 is a schematic block diagram of an example of a communicationbetween two or more of a user computing device (UCD) 14, an interactivecomputing device (ICD) 12, an interaction application server 20, ascreen-to-screen (STS) communication server 22, a payment processingserver 24, an independent server 26, and a cellular data base station130. In an example, the cellular data base station 130 is a networkportal (e.g., point-of-sale equipment, access point, internet protocol(IP) address, etc.).

In an example of operation, the servers 20-26, the cellular data basestation 130, the user computing device (UCD) 14 and the interactivecomputing device (ICD) 12 work in concert to exchange necessaryinformation to setup and execute a transaction via a screen to screen(STS) communication. For example, the UCD 14 downloads a userinteraction application from interaction application server 20 viacellular data base station 130 and the ICD 12 downloads a correspondingoperator interaction application from interaction application server 20via the cellular data base station 130. The UCD and the ICD utilizetheir respective interaction applications to assist in executing thetransaction.

During the transaction, the UCD 14 and the ICD 12 utilize the STScommunication path to wirelessly communicate individual data with eachother. The individual data includes one or more of personal data (e.g.,personal identification information, payment data, etc.), data that isconfidential at time of communication (e.g., a security code), data thatis particular to a transaction (e.g., payment information, selection ofitems information, etc.) and data that is meant only to be shared withone of or between the UCD 14 the ICD 12. As a specific example, a userselects items from a coffee shop user interaction application via atouch screen of UCD 14. The UCD 14 sends the selected items and paymentinformation to the ICD 12 via the STS communication.

The STS communication includes a medium for transmission and a datacommunication protocol. In an example, the medium is through a humanbody. In another example, the medium is through a close proximity (e.g.,<2 ft) of the UCD 14 and ICD 12. In a further example, the medium isthrough a surface of an object (e.g., store counter top, body, etc.).The data communication protocol indicates how the data is to becommunicated. For example the data communication protocol indicates whatmodulation scheme (e.g., amplitude shift keying, phase shift keying,frequency shift keying, amplitude modulation, 4 quadrature amplitudemodulation, etc.) and carrier signal (e.g., a sinusoidal signal having afrequency in the range of 10's of KHz to 10's of GHz) to use for the STScommunication.

Continuing with the example of operation of setting up and assisting thetransaction, the UCD 14 and ICD 12 each wirelessly communicate globaldata with the cellular data base station 130. In an example, the globaldata includes one or more of general data (e.g., account information,user preference information), setup data (e.g., update data, downloadingapplications), etc.), any data that is not the individual data, and anydata communicated between the cellular data base station 130 and the UCD14 and/or the ICD 12. As a specific example, the ICD 12 communicateswith payment processing server 24 to process the payment information.

FIG. 28 is a schematic block diagram of an example of a communicationthat is similar to FIG. 27, except in this example, a personal touchdevice 17 and a cell phone 19 is utilized instead of or as the usercomputing device 14. The personal touch device 17 is one or more of aFOB, a tablet, another cell phone, a car touch screen, a watch, a ring,and any wearable device.

In an example of operation, the personal touch device 17 and the cellphone 19 communicate personal data via a screen-to-screen (STS)communication. In an example, the personal data is the individual data.As another example, the personal data is a subset of the individualdata. As yet another example, the personal data is data that is moresensitive, private, and/or confidential than the individual data. As aspecific example, the personal data is the social security number (SSN)of a user and the individual data is the last four digits of the user'sSSN. As another specific example, the personal data is a password andthe individual data is a hash of the password. In another specificexample, the personal data is biometric information (facial recognition,fingerprint, voice frequency pattern, etc.) and the individual data is afour digit code (e.g., 7422). Note in this example, as illustrated bythe linear connection between the personal touch device 17 and the cellphone 19, the STS communication is a wired and/or wireless connection.

FIG. 29 is a schematic block diagram of an example of a communicationthat is similar to FIG. 27, except the personal touch device 17communicates directly with the interactive computing device (ICD) 12 viaa screen-to-screen (STS) communication. Note that although notexplicitly shown, the personal touch device 17 may communicate with thecell phone 19 via an STS wired and/or wireless connection. Further notethe connection provided by cellular data base station 130 may beimplemented by a network portal (e.g., point of sale equipment, accesspoint, internet protocol address, etc.).

In an example of operation, a communication is completed via acombination of an STS communication of individual data (e.g., personaldata for the particular transaction) between the personal touch device17 and the ICD 12 and a cellular data communication of global data(e.g., downloading applications, verifying user (e.g., of cell phone)and operator information (e.g., of ICD 12), etc.) between the cell phone19 and the cellular data base station 130, and between the ICD 12 andthe cellular data base station 130.

FIG. 30 is a schematic block diagram an example of a communication thatincludes an interactive computing device (ICD) 12, an interactionapplication server 20, a screen-to-screen (STS) communication server 22,a payment processing server 24, an independent server 26, a personaltouch device 17, a cell phone 19, and a cellular data base station 130.

In an example of operation, the personal touch device 17 interacts withcell phone 19 using a screen-to-screen (STS) communication (e.g., datacommunicated via an STS wired and/or wireless connection in accordancewith an STS communication protocol). For example, the personal touchdevice 17 communicates personal sensitive data (e.g., credit cardinformation, personal identity information, etc.) via the STScommunication to cell phone 19. The personal touch device 17 alsocommunicates a portion of interaction data (i.e., interaction data_1 ofa transaction) via another STS communication with the interactivecomputing device 12. The cell phone 19 communicates interaction data(i.e., interaction data_2 of the transaction) via another STScommunication with the interactive computing device 12.

As a specific example, the personal touch device is a hotel room keycard equipped with a radio frequency identification (RFID) tag and theinteractive computing device is a lock on a hotel room door. The lockrequires interaction data (e.g., interaction data_1 (e.g., first portionof a code) from the hotel room key card and interaction data (e.g.,interaction data_2 (e.g., second portion of the code)) from the cellphone 19 to perform an action (e.g., lock, unlock, display do notdisturb text, etc.). Note the code may indicate the action to beperformed. For example, a code of 7052 indicates an unlock function. Asanother example, a code of V3BH8 indicates to display a “do not disturb”image on a display of the lock. In a specific instance, the lockreceives the interaction data_2 from the cell phone 19 within atimeframe of receiving the interaction data_1 from the hotel room keycard to process the request.

As another specific example, the cell phone 19 is programmed (e.g., viaan STS communication application) to function a hotel room key (e.g.,key for “room 2455”) of a hotel. The hotel has numerous rooms that eachhave a lock on one or more doors that include an interactive computingdevice. For example, the lock is connected to an interactive computingdevice (ICD) that includes a touch screen. To unlock/lock the door, auser of the cell phone 19 may form an STS connection (e.g., via theuser's body as a network (BaaN)) with a touch screen of a particularinteractive computing device. For example, the touch screen of the ICDreceives a signal through the body of the user from the cell phone 19.This increases security as the personal touch device and cell phone bothmust interact with the ICD lock via an STS communication. For example,the user may lose its hotel key, but without cell phone 19, anunauthorized person (e.g., not the user) could not use the hotel key tooperable the ICD lock of the hotel room door.

The cell phone 19 subsequently transmits an STS communication thatinstructs (e.g., as a particular bit pattern and a certain frequency)the hotel room ICD lock to open. The lock may then automatically adjust(e.g., immediately upon closing, within a timeframe (e.g., 2 seconds)after closing, etc.) back to a lock position. Thus, a user is able tooperate the hotel room ICD lock more efficiently utilizing the STScommunication. For example, the user does not have to carry around anadditional “key”. As another example, the user can operate the ICD lockwithout removing the cell phone from their pocket (e.g., when using abody as a network (BaaN) STS connection).

FIG. 31 is a logic flow diagram of an example of a method, executed byan interactive computing device (ICD) and/or a user computing device(UCD) (hereinafter the ICD, the UCD and/or another computing device isreferred to as a computing device), of determining a type ofcommunication to use for an interaction between the user computingdevice and the interactive computing device. The determination is basedon one or more of a data type, a sensitivity (e.g., privacy level) ofthe data, a user application, an operator of interactive computingdevice, a bandwidth of the screen-to-screen connection and/or otherparameters.

The method begins with step 200, where the computing device initiates aninteraction (e.g., a communication of data between the UCD and the ICD).In an embodiment, the interaction includes a plurality of interactions(e.g., the interaction and other interactions). For example, a purchasea cup of coffee interaction includes an information exchange interaction(e.g., selection of items) and a purchase transaction interaction (e.g.,payment processing).

The method continues with step 202, where the computing devicedetermines an interaction type for each interaction. The interactiontype includes, but is not limited to, one or more of a one-way dataexchange, a two-way data exchange, a purchase transaction, aregistration transaction, a physical access transaction, an equipment(e.g., device, car, scooter, etc.) enable transaction, and a pre-paidtransaction.

The method continues to step 204, where for each interaction type, thecomputing device determines one or more data type(s). The one or moredata types include private information, publicly available information,payment information, transaction information, screen-to-screen (STS)communication account information, and user application accountinformation. The method continues to step 206, where the computingdevice determines available communication options. For example, theavailable communication options include a screen-to-screen (STS)communication, a cellular data communication, a Bluetooth communication,and wireless local area network (WLAN) communication.

The method continues to step 208, where the computing device determinesSTS communication capabilities of the UCD and the ICD. For example, thecomputing device determines whether the UCD and the ICD have one or moreof an STS communication unit 30 and an STS communication application. Asanother example, the computing device determines whether the UCD and theICD are able to form a body as a network (BaaN) connection. The methodcontinues to step 210, where the computing device determines data typecommunication restrictions. As a specific example, private informationis restricted (e.g., in accordance with a communication protocol) to aBaaN STS connection only, publicly available information is notrestricted, payment information is restricted to an STS connection only,transaction information is not restricted, however a first preference isfor it to be communicated via cellular data and a second (lesspreferential than the first preference) preference is for it to becommunication via a wireless local area network (WLAN), STScommunication account information is restricted to an STS connectionand/or cellular data only, and user application account information isrestricted from using WLAN.

The method continues to step 212, where for the data types to beutilized per interaction, the computing device determines whethercommunication options are available (e.g., unrestricted options exist).When communication options are available, the method continues to step214, where the computing device sets up the communications and theinteraction is executed. When communication options are not available,the method continues to step 216, where the computing device determineswhether other options are available. In an example, the other optionsare less desirable options but still allowable in accordance with therestrictions (e.g., transaction information communicated via a WLANconnection). When no other options are available, the method ends atstep 218. In an example, step 218 includes sending a message to the ICDand/or the UCD that indicates the interaction status (e.g., failed).When the other options are available, the method continues to step 220where the computing device makes changes to the communications. Forexample, the computing device changes the communication options fortransaction information from cellular to WLAN (e.g., less preferential),when WLAN is not against the restrictions for transaction information.

The method continues to step 222, where the computing device sets up thechanged communications. For example, the computing device instructs theICD and UCD to communicate transaction information via the WLANconnection. The method continues to step 224, where the computing deviceexecutes the interaction based on the changed communications. Forexample, the ICD and the UCD perform the interaction by sending thetransaction information via WLAN.

FIG. 32 is a schematic block diagram of an embodiment of initiating andsetting up screen to screen (STS) communications that includes a firstcomputing device (e.g., a user computing device 14) and second computingdevice (e.g., an interactive computing device 12). As illustrated,various communication types generally operate within a certain type ofrange (e.g., distance, signal strength, power level, size of body forbody as a network (BaaN) STS communications, etc.). For example,communications performed via a cellular network can be performed up to afourth range, communications performed via a wireless local area network(WLAN) can be performed up to a third range, communications performedvia Bluetooth can be performed up to a second range, and communicationsperformed via a screen to screen (STS) connection can be performed up toa first range, where the ranges descend (e.g., are less than, decrease,etc.) in order from the fourth to the first for at least one of thecertain types of range.

In an example of operation, the first computing device has a directionof movement 562. The direction of movement includes one or more of alocation, a direction, an altitude, a speed, a velocity, and anacceleration. For example, the direction of movement indicates the firstcomputing device is increasing elevation at 2.8 miles per hour in anorthwest direction. In an instance, a computing device (e.g., the firstcomputing device, the second computing device, another computing device,etc.) determines when/whether to setup or ready STS communicationabilities of the first computing device and/or the second computingdevice based on the direction of movement. For instance, when thedirection of movement of the first computing device is toward the secondcomputing device such that it is estimated that the first computingdevice will be inside an STS communication range within a first timeperiod, a STS communication readiness check is initiated.

As an example, when the first computing device has a first trajectoryand a first spatiotemporal quality (e.g., a first distance from an ICD,a first estimated time from being within a range of the ICD, etc,) thefirst computing device is prompted to perform a first action (e.g.,download an STS communication application, pre-order a typical orderassociated with an application regarding the second computing device,etc.). As another example, when the first computing device has the firsttrajectory and the first spatiotemporal quality, the second computingdevice is instructed to perform a first action (e.g., begin preparing anorder for the customer, ensure customer database is updated withinformation of a user associated with the first computing device, updateapplication on a computing device, etc.).

The direction of movement 562 may further determine which type ofcommunications to use. For example, the first and second computingdevices determine to communicate via WLAN for a first time period and/oruntil the first computing device is within range of anothercommunication type (e.g., Bluetooth, STS, etc.).

FIG. 33 is a logic flow diagram of another example of a method ofsetting up a screen to screen (STS) communications between aninteractive computing device (ICD) and a user computing device (UCD).Note that hereinafter in the discussion of this figure the ICD, the UCDand/or another computing device (e.g., an interactive server, an STScommunication server, etc.) are referred to as a computing device. Themethod begins at step 300, where the computing device determines whetherthe UCD is inside a local communication range. The local communicationrange includes one or more of a wireless local area network (WLAN)range, a cellular data network range, a Bluetooth connection range, andan STS connection range.

In an example, the UCD periodically or continually searches for awireless local area network (WLAN) associated with the ICD to determinewhether the UCD is within the WLAN range. As another example, thecomputing device determines a distance (e.g., using global positioningsystem (GPS) data and/or direction of movement data) between the ICD andthe UCD to determine whether the UCD is within a STS communication range(or a likelihood of the UCD coming within range during a time period).As a specific example, the computing device utilizes the distance of theUCD and the ICD to determine whether the UCD is in line inside a coffeeshop or in a drive thru lane of the coffee shop. When the UCD is notinside the local communication range, the method continues back to step300.

When the UCD is inside the local communication range, the methodcontinues to step 302, where the computing device determines whether toset up the local communication(s). When not setting up the localcommunication, the method continues back to step 300. When setting upthe local communication, the method continues with step 304, where thecomputing device sends a query to the UCD to determine whether the UCDhas screen to screen (STS) communication software (e.g., application)installed and/or accessible. In an example, the query also asks whetherthe UCD has STS communication hardware (e.g., a drive sense module, atouch screen with an electrode, etc.).

The method continues with step 306, where the computing devicedetermines (e.g., based on a query response) whether the UCD has the STScommunication application. When the UCD does not have the STScommunication application, the method continues to step 308, where theUCD obtains the STS communication application via one or morecommunication networks (e.g., a wide area network (WAN), a local areanetwork (LAN), cellular data network (e.g., 5G), etc.). For example, theUCD downloads the STS communication application from an STScommunication server via a 5G cellular data network connection.Alternatively at step 308, or in addition to, when the UCD doesn'tdownload (e.g., can't download, determines not to download, etc.) theSTS communication application, the process ends and/or the computingdevice sends a message to the UCD for the user to go inside and interactwith an ICD for further instructions.

The method continues with step 310, where the computing device sends aquery to the UCD to determine whether the UCD has an interactive userapplication installed or accessible. The method continues to step 312,where the computing device determines (e.g., based on a query response)whether the UCD has the interactive user application. When the UCD doesnot have the interactive user application, the method continues to step314, where the UCD obtains (e.g., downloads, gain access to, etc.) theinteractive user application via one or more of the communicationnetworks (e.g., a wireless area network (WAN)). Alternatively, or inaddition to, when the UCD doesn't download (e.g., can't download,determines not to download, etc.) the interactive user application, theprocess ends and/or the computing device sends a message to the UCD forthe user to go inside and interact with an ICD for further instructions.The method then continues to step 316. When the UCD has the interactiveuser application, the method continues to step 316, where the UCD andICD execute a transaction at least partially via an STS communicationlink.

FIG. 34 is a logic flow diagram of another example of a method ofsetting up a screen to screen (STS) communication between an interactivecomputing device (ICD) and a user computing device (UCD). As used in thedescription of this figure, the ICD, the UCD and/or another computingdevice are referred to as a computing device. The method begins orcontinues with step 340, where the computing device determines whether aUCD is inside a local communication range (e.g., 5G, wireless local areanetwork, wide area network, Bluetooth, etc.). When not inside the localcommunication range, the method continues back to step 340. When insidethe local communication range, the method continues to step 342, wherethe computing device determines whether it can set up a localcommunication.

When the local communication cannot be setup, the method continues backto step 340. When the local communication can be setup, the methodcontinues to step 344, where the computing device determines whether theUCD has an STS communication application installed and/or accessible.For example, the computing device queries the UCD to respond with anindication of whether it has the STS communication application. When theUCD does not have the STS communication application, the methodcontinues to step 345, where the UCD gets the STS communicationapplication. Alternatively, when the UCD does not get the STScommunication application, the process ends. When the UCD has the STScommunication application, the method continues to step 346, where thecomputing device determines whether to pre-order (e.g., via aninteraction application) one or more items via a local communicationnetwork (e.g., 5G, WLAN of a coffee shop).

When the computing device determines not to pre-order one or more itemsvia the local communication, the method continues to step 347, where thecomputing device determines to wait until a user of a UCD is at aninteractive computing device (e.g., of the coffee shop) to order via ascreen to screen (STS) communication. When the computing devicedetermines to pre-order one or more items via the local communication,the method continues to step 348, where the computing device places apre-order of the one or more items via a local communication link. Forexample, the user computing device sends a message to an ICD (or othercomputing device (e.g., coffee shop server)) of a coffee shop thatincludes data regarding a coffee order (regular order, particular orderbased on a day of a week and/or time of the day, etc.). The methodcontinues with step 349, where the computing device finalizes the order(e.g., provides payment data, provides signature, selects reward pointsas payment, etc.) via a screen to screen (STS) communication between theUCD and the ICD.

FIG. 35 is a schematic block diagram of an example of transmitting closeproximity signals 127 between a user computing device (UCD) 14 and aninteractive computing device (ICD) 12 to form a screen to screen (STS)connection 18. In this example, the user computing device 14 may or maynot include a display associated with the touch screen sensor array 34and the interactive computing device does include a display associatedwith its touch screen sensor array 34.

In an example of operation, a user (e.g., of UCD 14) touches a button(e.g., start) on a touch screen of the ICD 12 to initiate setting upscreen to screen (STS) communications (e.g., how the ICD and UCD willinteract in a transaction that includes at least some data transmittedbetween the ICD and UCD over an STS connection). Alternatively, the usermay touch a portion of the UCD 14 touch screen to initiate setting upthe STS communications. The ICD 12 transmits a signal (e.g., a defaultping signal) to the UCD 14 to initiate an STS connection via closeproximity 127 and/or body as a network (BaaN). The UCD receives the pingsignal and sends a ping back signal to the ICD. The ping signal and pingback signal are discussed in further detail with reference to one ormore subsequent figures.

FIG. 36 is a schematic block diagram of an example of transmitting pingsignals via a body as a network (BaaN) screen to screen (STS)connection. As illustrated, the interactive computing device (ICD) 12and the user computing device (UCD) 14 each include drive sense modules(DSMs) connected to rows of electrodes 105 and columns of electrodes105. When a user touches the touch screen, the drive sense modules sensethe touch based on a change in an electrical characteristic (e.g., animpedance, a current, a reactance, a voltage, a frequency response,etc.) of the affected electrodes 105 at one or more particularfrequencies (e.g., fs, fm_1 to fm_n of FIG. 32). The drive sense modulesalso sense a ping signal at another one or more particular frequencies(e.g., f1, f2, f3 of FIG. 32) of the affected electrodes 105.

Based on the detected touch, the touch screen processing modulesdetermine to drive a signal onto the affected electrodes as a method oftransmitting data via the STS connection. For example, the ICD 12 sensesa ping signal at a first frequency (f1) on an electrode. The ICD drivesa ping back signal onto the electrode at f1 and/or another frequency.

FIG. 37 is a schematic block diagram of an example of an interactivecomputing device (ICD) 12 generating a default ping signal andtransmitting the default ping signal via electrodes that are affected bya user touch. In this example, the ICD 12 creates the default pingsignal that is to be transmitted to a user computing device (UCD) viathe affected electrodes 105. The signal may be generated in accordancewith a modulation scheme. For example, the ICD utilizes an amplitudemodulation (AM) scheme to produce the default ping signal. As anotherexample, the ICD utilizes an amplitude shift keying modulation scheme toproduce the default ping signal. When the ICD utilizes AM or ASK, areceiving device is able to determine the default ping signal withoutsyncing the UCD's clock with a clock of the ICD.

FIG. 38 is a schematic block diagram of an example of a default pingsignal. The default ping signal is generated at one or more particularfrequencies (e.g., 300 cycles per second, 300 MHz, 1 GHz, etc) and isrepeated in accordance with a screen to screen (STS) communicationprotocol. The default ping signal indicates to another computing deviceto setup an STS communication.

In this example, the default ping signal is 16 cycles using a two-levelencoding. For example, the ICD transmits at no frequency or a firstfrequency in accordance with an on-off keying (OOK) modulation scheme,which represents the binary equivalent of 1 bit per cycle. When the ICDdoes not transmit the first frequency (e.g., no TX) during a cycle, thisrepresents a binary 0. And, when the ICD transmits the first frequencyduring a cycle, this represents a binary 1. However, other embodimentsmay use more or less than 16 cycles, more than 1 frequency, and/or morebits per cycle (e.g., four level encoding scheme to represent two bitsper cycle as illustrated in FIG. 39).

For example, a default signal has a pattern of 8 cycles at a firstfrequency. As another example, a default ping signal has a pattern of 8cycles at the first frequency and eight cycles at a second frequency. Asa further example, a default ping signal has a pattern of 4 cycles atthe first frequency and 4 cycles at no frequency, 4 cycles at the secondfrequency, 2 cycles at the no frequency and 2 cycles at the secondfrequency. As yet another example, a default ping signal has a patternthat repeats three total cycles of 8 cycles at the first frequency and 8cycles at the second frequency. Note that the frequencies used in thedefault ping signal may be dedicated for the ping signal. Alternatively,or in addition to, the frequencies used in the default ping signal maybe different from frequencies utilized to determine self and/or mutualcapacitance of the electrodes.

FIG. 39 is a schematic block diagram of an example of transmitting adefault ping signal. In this example, the default ping signal istransmitted in a pattern of 16 cycles using no frequency and a first,second and third frequency, each of which have a binary equivalent oftwo bits (e.g., 00, 01, 10, 11). The pattern may be repeated a certainnumber of times according to a screen to screen (STS) communicationprotocol to ensure a receiving computing device can receive and identifythe default ping signal.

FIG. 40 is a schematic block diagram of an example of transmitting adefault ping signal shown in FIG. 39 via an electrode 105 that isconnected to a front end of a drive sense circuit 103. The front end ofthe drive sense circuit 103 includes a current source 111 and acomparator 112 connected to the electrode 105. The comparator isinputted an analog reference signal 101 which it uses to compare tosignaling on the line connected to the electrode 105 and dependentcurrent source 111.

An example of the analog reference signal 101 is shown having a directcurrent (DC) component 324 that has a magnitude and an oscillatingcomponent 326 oscillating at a frequency “i”. The output of thecomparator changes in part based on changes to analog reference signal101. For example, a processing module of an interactive computing devicemodulates data onto a carrier signal at none, a first, a second, and athird frequency to produce the analog reference signal 101 (e.g., f“i”).The comparator generates an analog compensation signal based on thechanges to the analog reference signal. The current source 111 modifies(e.g., increases, decreases) an output current based on the analogcompensation signal, that is driven onto electrode 105. An electricalcharacteristic of the electrode is affected by the output current and isrepresentative of the modulated data (e.g., transmitting no signal,transmitting a signal at a first frequency (e.g., f1), transmitting asignal at a second frequency (e.g., f2) and transmitting a signal at athird frequency (e.g., f3)).

FIG. 41 is a logic flow diagram of an example of a method forestablishing a screen to screen (STS) connection. FIG. 42 is a schematicblock diagram illustrating the affected electrodes 105 of an interactivecomputing device (ICD) 12 as discussed in the example of FIG. 41. Themethod of FIG. 41 begins with step 360, where an interactive computingdevice (ICD) detects a touch by a user on a touch screen of the ICD. Themethod continues with step 362, where the ICD determines the electrodesaffected by the user touch. For example, a processing module of the ICDdetermines a change in the self and/or mutual capacitance of electrodesthat are affected by the user touch and interprets the change ofcapacitance as representing a touch. The touch may include two or moretouch points (e.g., different affected electrodes).

The method continues with step 364, where the ICD creates a defaultscreen to screen (STS) ping signal. For example, the ICD generates asignal with a particular frequency pattern that represents a ping signalin accordance with a STS communication protocol. The method continueswith step 366, where the ICD transmits the default STS ping signal viathe affected electrodes (e.g., the bolded electrodes of FIG. 42). Themethod continues to step 368, where the ICD determines whether the useris still touching the touch screen (e.g., at least a portion of theaffected electrodes, any electrodes of the touch screen, etc). When theuser is not touching the touch screen, the method continues to step 369,where the ICD generates a message instructing the user to touch thetouch screen again and hold until next steps. Alternatively, the methodends at step 369.

When the user is still touching the touch screen, the method continuesto step 370, where the ICD determines whether it has received a pingback signal (e.g., from a user computing device of the user). When theICD has not received the ping back signal (e.g., within a time frame),the method continues back to step 366. Alternatively, when the ICD hasnot received the ping back signal, the method may end, or continue tostep 368. When the ICD has received the ping back signal, the methodcontinues to step 372, where the ICD establishes a type (e.g., closeproximity, via human body, etc.) of STS connection. For example, the ICDestablishes the STS connection is via a human body (e.g., body as anetwork (BaaN)). Note the type of connection (e.g., close proximity) forthe STS may be different than a type of connection (e.g., BaaN) utilizedto setup the STS communications.

FIG. 43 is a schematic block diagram of an example of receiving adefault ping signal by a user computing device (UCD) 14. In thisexample, the default ping signal includes 16 cycles of either notransmission (which represents a binary 0) or transmission at a firstfrequency (which represents a binary 1). The UCD 14 receives the defaultping signal via a body of a user that is touching (or close enough totransmit the default ping signal) a touch screen of the UCD 14 and atouch screen of an interactive computing device (ICD) 12.

FIG. 44 is a schematic block diagram of an example of receiving a pingsignal 231 on an electrode 105 connected to a front end of a drive sensecircuit 103 (e.g., of a user computing device 14). The front endincludes a current source 111 and a comparator 112.

In an example of receiving the ping signal 231, the comparator 112compares an analog reference signal 101 (e.g., a current signal or avoltage signal) to an electrode signal 321 to produce an analogcomparison signal 325, which represents a change in an electricalcharacteristic of the electrode 105. The received ping signal 231includes a direct current (DC) component 320 and an oscillatingcomponent 322. The DC component 320 is a DC voltage in the range of afew hundred milli-volts to tens of volts or more. The oscillatingcomponent 322 includes a sinusoidal signal, a square wave signal, atriangular wave signal, a multiple level signal (e.g., has varyingmagnitude over time with respect to the DC component), and/or apolygonal signal (e.g., has a symmetrical or asymmetrical polygonalshape with respect to the DC component).

The oscillating component 322 oscillates at a frequency “f_(i)”. In anexample, f_(i) includes one or more of a first frequency (f1), a secondfrequency (f2) and a third frequency (f3) (e.g., as illustrated in themagnitude frequency graph of the ping signal). In this example, thefirst, second, and third frequencies are the frequencies utilized tosetup screen to screen (STS) communications between devices. As anotherexample, fi is a carrier frequency. As another example, fi is thecombination of the carrier signal that is modulated with data signals atone or more frequencies (e.g., f1, f2, f3).

The analog reference signal 101 includes a DC component 324 and anoscillating component(s) for self and/or mutual capacitance 326. As anexample, the oscillating component(s) include a frequency (fs) fordriving/sensing a self-capacitance of an electrode and one or morefrequencies (fm_1 to fm_n) for driving/sensing mutual capacitancesbetween the electrode and other electrodes. The frequencies of selfand/or mutual capacitances of a touch screen are utilized to determinewhich electrodes are touched (e.g., affected electrodes), and/or how atouch screen is touched (e.g., motion, etc.) and further what istouching it (e.g., pen, human finger, etc.). For example, the drivesense modules that detect capacitance changes and the type ofcapacitance change (e.g., self, mutual) are utilized to determine whichelectrodes of the touch screen are affected by the touch.

Continuing with the example, the current source modifies a current basedon the analog comparison signal to keep a voltage on the electrodesubstantially constant. A processing module determines the presence off1, f2, and/or f3 based on the analog comparison signal 325. Theprocessing module further determines whether the analog comparisonsignal 325 indicates the user computing device is receiving a pingsignal (e.g., default bit pattern) from another computing device (e.g.,an interactive computing device 12).

FIG. 45 is a schematic block diagram of an example of generating a pingback signal via an electrode 105 that is connected to a comparator 112and a current source 111. As illustrated, the comparator is inputted ananalog reference signal 101 and signaling on a line connected to anoutput of the current source 111 and the electrode 105. The analogreference signal 101 includes a direct current (DC) component 324 and anoscillating component 327 that oscillates at a frequency “k”.

The comparator 112 outputs an analog compensation signal based on acomparison of the analog reference signal and signaling on electrode105. The current source 111 adjusts a current based on the analogcompensation signal to keep the inputs of the comparator substantiallythe same (e.g., same voltage, same current). The electrode transmits theping back signal based on the current adjustment (e.g., current drivenon electrode 105) at one or more frequencies and/or the currentadjustment based on the received ping signals.

In this example, when the electrode is effectively transmitting (at asecond frequency) while receiving a signal (e.g., at a first frequency),the ping back signal (shown in green) oscillates based on a firstfrequency component (e.g., f“i”) and a second frequency component (e.g.,f“k”). For example, the signal component f“i” is combined (e.g., added,multiplied) with the signal component f“k” to produce the ping backsignal.

FIG. 46 is a schematic block diagram of an example of producing a pingback signal that includes a current source 111, a comparator 112, anelectrode 105, a bandpass filter 454, and a modulator 452. Alsoillustrated are a time domain graph that plots magnitude versus time fora ping back signal using amplitude shift keying (ASK), and a frequencydomain graph that plots magnitude versus frequency of the ping backsignal.

In an example of operation, the comparator 112 outputs an analogcomparison signal based on its inputs. For example, the electrodereceives a default ping signal that changes an electrical characteristicof the electrode. The comparator outputs the analog comparison signalsuch that it represents a signal component of the default ping signal.The bandpass filter 454, filters out unwanted frequencies to produce arecovered signal component at a desired frequency (e.g., f “i”). Themodulator 452 modulates the recovered f“i” signal component based onping back data 450 to produce a ping back reference input. Themodulation includes one or more of amplitude shift keying (ASK),amplitude modulation (AM), phase shift keying (PSK), and 4-quadratureamplitude modulation (4QAM).

The comparator produces a second analog comparison signal based on theping back reference input, which causes current source 111 to adjust acurrent signal to keep the inputs to the comparator substantiallyconstant. The current signal is driven onto electrode 105 to produce aping back signal that represents ping back data 450.

FIG. 47 is a logic flow diagram of an example of a method of setting upa screen to screen (STS) connection. In this example, the STS connectionis between an interactive computing device (ICD) and a user computingdevice (UCD). However, in other examples, the screen to screenconnection is set up between one or more UCDs and/or one or more ICDs.

The method begins at step 400, where the interactive computing device(ICD) provides an on-screen “start” button. The “start” button may be aphysical button to press, a representation of a button on the display ofa touch screen of the ICD, and/or an instruction (e.g., text, voice,etc.) to place a user computing device in a particular area, such thatthe user computing device is orientated with respect to the ICD toenable an STS connection. In an example, the button (or additionalbutton) further includes an indication of the STS connection type touse. For example, a first button indicates to use a close proximityconnection and a second button indicates to use a human body connection.In another example, the ICD includes another mechanism (e.g., physicalbutton, prompt to complete a Completely Automated Public Turing test totell Computers and Humans Apart, (CAPTCHA), another digital button, amotion, a voice command, etc.). that ensures it is the intent of theuser to start the STS connection process.

The method continues with step 402, where the ICD determines whether auser touch has been detected. When the user touch has not been detected,the method continues back to step 400. When the user touch has beendetected, the method continues to one or more of steps 403 and 404. Atstep 403, the ICD displays an instruction to touch a portion of the ICDtouch screen (e.g., a touch here button) while the user is touching(e.g., body is in contact with) the user computing device (UCD). At step404, the ICD displays an instruction to place the UCD in an area of oradjacent to the ICD display, such that a close proximity or vibrationSTS connection is able to be formed.

After steps 403 and/or 404, the method continues to step 406, where theICD sends an STS ping signal to the UCD. The STS ping signal is adefault signal for any type of STS connection or is a first particularsignal for a first STS connection type and a second particular signalfor a second STS connection type. The method continues to step 408,where the ICD determines whether it has received a ping back signal.During step 408, the user computing device is actively looking for theSTS ping signal from the ICD. An example of the UCD looking for the STSping signal is discussed in further detail with reference to FIG. 48.

When the ICD has not received the ping back signal within a time period,the method continues to step 410, where the ICD determines whether thewait (e.g., elapsed time) looking for the ping signal has expired (e.g.,timed out). When the ICD determines the wait for the receive ping signalhas timed out, the method continues to step 412, where the ICD ends theprocess. Alternatively, or in addition to, the ICD may display a messageto download an STS communication application on the UCD, a message tostart over with the user, and/or a reminder message of an action to take(e.g., place hand on screen, place phone on screen, touch physicalbutton on side of ICD, etc.). When the ICD determines the wait for thereceive ping signal has not timed out, the method continues back to step406, where the ICD sends another STS ping signal to the user computingdevice.

When the ICD has received the ping back signal within the time period,the method continues to step 414, where the ICD and the UCD establish atype of STS connection. For example, the ICD and UCD establish toperform STS communication via close proximity STS connection. As anotherexample, the ICD and UCD establish to perform STS communication via theuser's body as a network (BaaN) STS connection.

Having established the type of STS connection, the method continues withstep 416, where the ICD and UCD establish an STS communication protocolfor the STS communication. For example, the STS communication protocolestablishes STS communications are to be in accordance with a particulartype of one of pattern encoding, binary encoding, and symbol encoding.

FIG. 48 is a logic flow diagram of another example of a method for usein setting up a screen to screen (STS) connection between an interactivecomputing device (ICD) and a user computing device (UCD). The methodbegins with step 420, where a UCD periodically senses for an STS pingsignal. For example, the UCD has an STS communication applicationinstalled or otherwise able to access and the STS communicationapplication periodically wakes up to listen or is always listening forthe STS ping signal.

The method continues with step 422, where the UCD determines whether ithas detected an STS ping signal. When the STS ping signal is notdetected, the method continues back to step 420. When the STS pingsignal is detected, the method continues to step 424, where the UCDtransmits a ping back signal. In an example, the ping back signal is aring back signal.

The method continues with step 426, where the ICD and the UCD establisha type of STS connection. For example, the ICD and UCD establish toperform STS communications via a close proximity STS connection. Asanother example, the ICD and UCD establish to perform STS communicationsvia the user's body as a network (BaaN) STS connection.

Having established the type of STS connection, the method continues withstep 428, where the ICD and UCD establish an STS communication protocolfor the STS communication. For example, the STS communication protocolestablishes STS communications are to be in accordance with one ofpattern encoding, binary encoding, and symbol encoding.

FIG. 49 is a logic flow diagram of another example of a method ofsetting up a screen to screen (STS) connection between an interactivecomputing device (ICD) and a user computing device (UCD). The methodbegins with step 430, where the user computing device detects a touch ofthe screen by the user. For example, a touch screen processing module ofthe UCD interprets a capacitance (e.g., self-capacitance, mutualcapacitance) change of one or more electrodes of the touch screen of theUCD to determine the touch.

The method continues with step 432, where the UCD determines electrodesaffected by the touch. For example, the UCD determines which drive sensemodules that are coupled to the electrodes (e.g., coupled to anelectrode, a row of electrodes, a column of electrodes, etc.) detected acapacitance change at a certain frequency to determine the affectedelectrodes. The method continues with step 434, where the UCD receives adefault ping signal via the affected electrodes.

The method continues with step 436, where the UCD determines whether itrecognizes a pattern (e.g., transmission cycle pattern, frequencypattern, an amplitude pattern, etc.) of the default ping signal as thedefault ping signal. When the UCD does not recognize the pattern, themethod continues to step 438, where the UCD determines whether the useris still touching the UCD touch screen. When the user is still touching,the method continues to step 434. When the user is not still touching,the method continues to step 439, where the UCD ends the process.Alternatively, the UCD prompts the user to touch the screen again andhold until the STS communication is setup or until the UCD prompts theuser that it is ok to stop touching the UCD touch screen.

When the UCD does recognize the pattern, the method continues to step440, where the UCD generates a ping back signal. In an example, the UCDbackscatters the default ping signal or pings back the signal pattern(e.g., inverse of the ping signal, same pattern as ping signal, etc.).The method continues with step 442, where the UCD transmits a ping backsignal. The method continues with step 444, where the UCD determineswhether it has received an acknowledgement from the ICD.

When the UCD has not received the acknowledgement, the method continuesto step 445, where the UCD determines whether a time period forreceiving the acknowledgement has ended (e.g., the process times out).When the process has not timed out, the method continues to step 442.When the process has timed out, the method continues to step 446, wherethe UCD ends the process. In addition, the UCD may ask the user to startthe STS connection process over and/or ask the user to repeat touchingthe touchscreen so that the UCD can retry sending the ping back signal(e.g., step 442) to the ICD.

When the UCD has received the acknowledgement (ACK), the methodcontinues to step 448 where the UCD and/or ICD establishes the type ofSTS connection. For example, the ICD and UCD establish to perform STScommunication via close proximity STS connection. As another example,the ICD and UCD establish to perform STS communication via the user'sbody as a network (BaaN) STS connection.

FIG. 50 is a schematic block diagram of an embodiment of an example of aradio frequency (RF) transceiver 460 and a signal source 102, and anillustration of the output of the signal source 102 (e.g., analogreference signal 101). The RF transceiver 460 includes a digitalbaseband or low IF processing module 461, an analog to digital converter(ADC) 450, a receive (RX) low pass (LP) filter circuit 462, downconversion mixer 463, a low noise amplifier 464, a receive (RX) bandpass(BP) filter circuit 465, a transmit (TX)/receive (RX) splitter 466coupled to an antenna, a transmit (TX) bandpass (BP) filter circuit 467,a power amplifier 468, an up conversion mixer 469, a transmit low pass(LP) filter circuit 470, a digital to analog converter (DAC) 452 and alocal oscillation generator (LOGEN) 473. The signal source 102 includesa direct current (DC) reference voltage circuit 471, a phase locked loop(PLL) 472, and a combining circuit 474.

In an example of operation, the antenna of the TX/RX splitter 466 (e.g.,a balun, a duplexer, circulator, etc) receives an inbound radiofrequency (RF) signal, which is routed to the RX BP filter module 465.The RX BP filter module 465 is a filter that passes the inbound RFsignal to the LNA 464, which amplifies the inbound RF signal to producean amplified inbound RF signal.

The down conversion mixer 463 converts the amplified inbound RF signalinto an inbound symbol stream corresponding to a first signal componentand into a second inbound symbol stream corresponding to the secondsignal component. In an embodiment, the down conversion mixer 463 mixesin-phase (I) and quadrature (Q) components of the amplified inbound RFsignal with in-phase and quadrature components of local oscillationgenerator 473 to produce a mixed I signal and a mixed Q signal for eachcomponent of the amplified inbound RF signal. Each pair of the mixed Iand Q signals are combined to produce the first and second inboundsymbol streams. In this embodiment, each of the first and second inboundsymbol streams includes phase information (e.g., +/−Δθ [phase shift]and/or θ(t) [phase modulation]) and/or frequency information (e.g.,+/−Δf [frequency shift] and/or f(t) [frequency modulation]). In anotherembodiment, the inbound RF signal includes amplitude information (e.g.,+/−ΔA [amplitude shift] and/or A(t) [amplitude modulation]). The RX LPfilter circuit 462 filters the down-converted inbound signal, which isthen converted into a digital inbound baseband signal by the ADC 450.

The digital baseband or low IF processing module 461 converts theinbound symbol stream(s) into data in 453 (e.g., voice, text, audio,video, graphics, etc.) in accordance with one or more wirelesscommunication standards (e.g., GSM, CDMA, WCDMA, HSDPA, HSDPA, WiMAX,EDGE, GPRS, IEEE 802.11, Bluetooth, ZigBee, universal mobiletelecommunications system (UMTS), long term evolution (LTE), IEEE802.16, evolution data optimized (EV-DO), etc.). Such a conversion mayinclude one or more of: digital intermediate frequency to basebandconversion, time to frequency domain conversion, space-time-blockdecoding, space-frequency-block decoding, demodulation, frequency spreaddecoding, frequency hopping decoding, beamforming decoding,constellation demapping, deinterleaving, decoding, depuncturing, and/ordescrambling. Note that the processing module 461 converts a singleinbound symbol stream into the inbound data for Single Input SingleOutput (SISO) communications and/or for Multiple Input Single Output(MISO) communications and converts the multiple inbound symbol streamsinto the inbound data for Single Input Multiple Output (SIMO) andMultiple Input Multiple Output (MIMO) communications.

In this example, the processing module 461 receives data out 455. As anexample, the processing module interprets the data out 455 as a touch ofa touch screen to generate a command (e.g., pause, stop, etc.) regardinga streaming video. The processing module processes the command byconverting it into one or more outbound symbol streams (e.g., outboundbaseband signal) in accordance with one or more wireless communicationstandards (e.g., GSM, CDMA, WCDMA, HSDPA, HSDPA, WiMAX, EDGE, GPRS, IEEE802.11, Bluetooth, ZigBee, universal mobile telecommunications system(UMTS), long term evolution (LTE), IEEE 802.16, evolution data optimized(EV-DO), etc.). Such a conversion includes one or more of: scrambling,puncturing, encoding, interleaving, constellation mapping, modulation,frequency spreading, frequency hopping, beamforming, space-time-blockencoding, space-frequency-block encoding, frequency to time domainconversion, and/or digital baseband to intermediate frequencyconversion. Note that the processing module converts the outbound datainto a single outbound symbol stream for Single Input Single Output(SISO) communications and/or for Multiple Input Single Output (MISO)communications and converts the outbound data into multiple outboundsymbol streams for Single Input Multiple Output (SIMO) and MultipleInput Multiple Output (MIMO) communications.

The DAC 452 converts the outbound baseband signal into an analog signal,which is filtered by the TX LP filter circuit 470. The up-conversionmixer 469 mixes the filtered analog outbound baseband signal with atransmit local oscillation (TX LO) to produce an up-converted signal.This may be done in a variety of ways. In an embodiment, in-phase andquadrature components of the outbound baseband signal are mixed within-phase and quadrature components of the transmit local oscillation toproduce the up-converted signal. In another embodiment, the outboundbaseband signal provides phase information (e.g., +/−Δθ [phase shift]and/or θ(t) [phase modulation]) that adjusts the phase of the transmitlocal oscillation to produce a phase adjusted up-converted signal.

In this embodiment, the phase adjusted up-converted signal provides theup-converted signal. In another embodiment, the outbound baseband signalfurther includes amplitude information (e.g., A(t) [amplitudemodulation]), which is used to adjust the amplitude of the phaseadjusted up converted signal to produce the up-converted signal. In yetanother embodiment, the outbound baseband signal provides frequencyinformation (e.g., +/−Δf [frequency shift] and/or f(t) [frequencymodulation]) that adjusts the frequency of the transmit localoscillation to produce a frequency adjusted up-converted signal. In thisembodiment, the frequency adjusted up-converted signal provides theup-converted signal. In another embodiment, the outbound baseband signalfurther includes amplitude information, which is used to adjust theamplitude of the frequency adjusted up-converted signal to produce theup-converted signal. In a further embodiment, the outbound basebandsignal provides amplitude information (e.g., +/−ΔA [amplitude shift]and/or A(t) [amplitude modulation) that adjusts the amplitude of thetransmit local oscillation to produce the up-converted signal.

The power amplifier (PA) 468 amplifies the up-converted signal toproduce an outbound RF signal. The TX BP filter circuit 467 filters theoutbound RF signal and provides the filtered outbound RF signal to theTX/RX splitter 466 for transmission via the antenna that is connected tothe TX/RX splitter 466.

The LOGEN 473 also provides a reference oscillation signal to a phaselocked loop (PLL) 472 of the signal source 102. The phase locked loop472 locks onto a phase and/or frequency of the reference oscillationsignal to produce an oscillating component 322. Note the frequency ofthe oscillating component may be different (e.g., greater than, lessthan) than a frequency of the reference oscillation signal. Further notein an example, the PLL is omitted and the LOGEN 473 provides theoscillating component 322 to the combining circuit 474.

The direct current (DC) reference voltage circuit 471 produces a directcurrent (DC) component 320. The combining circuit 474 combines (e.g.,adds, multiples, etc.) the oscillating component 322 and the DCcomponent 320 to produce analog reference signal 101.

FIG. 51 is a schematic block diagram of an example of an interactivecomputing device (ICD) interacting with a user computing device (UCD) toselect items. As illustrated, a menu of 9 items are displayed on theICD. Note the menu may include for each corresponding item one or moreof a graphical representation, nutritional information, priceinformation, ingredient information, and estimated completion timeinformation. Further note a running total of a user's selections couldalso be displayed. For example, a sidebar of the menu displays items auser has already selected along with a total purchase price (e.g., in acurrency (e.g., dollars, pounds, bitcoin, etc.) and/or rewards elements(e.g., points, stars, rewards level, etc.).

In an example of operation, the ICD provides (e.g., displays, sends to auser computing device (UCD)) a menu of options able to be selected by auser. The ICD receives one or more selections of options via a touch(e.g., BaaN) from the user on the touch screen of the ICD, a voiceselection from the user, a Bluetooth communication from the UCD, and/orin combination with an STS communication.

FIG. 52 is a schematic block diagram of an example of an interactivecomputing device (ICD) interacting with a user computing device (UCD) tomirror a menu of items. In an example of operation, the user of a UCDopens a coffee company application on the UCD. The coffee application ismirrored, at least partially, on an ICD associated with the coffeecompany (e.g., point of sale (POS) at brick and mortar location). Themirroring may be performed via a wireless local area network (WLAN),Bluetooth and/or a cellular data network (e.g., 5G network).

In an example, the UCD and ICD have already set up an STS connection(e.g., via user touching the ICD, via user placing the UCD in closeproximity to the ICD, etc.). In another example, the UCD and ICD willsetup an STS connection during or subsequent to the selection of menuitems. As illustrated, the user selects item 2 on a touch screen of theUCD and the ICD displays a mirrored menu showing item 2 being selected.

FIG. 53 is a schematic block diagram of an example of an interactivecomputing device (ICD) interacting with a user computing device (UCD) toselect items of a menu. The UCD displays a menu of items for selectionsby a user. The menu may be part of an application downloaded on the UCD.The application may be acquired via the ICD and/or from an interactionapplication server 20.

In this example, the user computing device (UCD) receives selections ofthe menu from a user via its touch screen. For example, the user touchesan area of the touch screen that corresponds to a selection of item 2.In an embodiment, the user touches the area (“button”) of the touchscreen that displays an item a certain number of times (e.g., releasingfinger and then placing finger in same area again) corresponding to adesired quantity of the item. As a specific example, when the userdesires two lattes and one breakfast sandwich, the user touches thebutton for a latte twice and the breakfast sandwich once. In anotherembodiment, after the user makes a selection (e.g., touches item 2), aquantity selection option (e.g., in same area of as item 2 on the touchscreen, in different area of touch screen, etc.) is then displayedprompting the user to input a quantity or confirm a default (e.g., 1)quantity.

Having received the selection of an item, the UCD sends the selectionsto the ICD, which displays the selections on a display of the ICD. Forexample, the user selects a quantity of two of item 7, a quantity of oneof item 4, and a quantity of three of item 2. As illustrated, the ICDmay display the selections along with price information.

FIG. 54 is a schematic block diagram of an example of an interactivecomputing device (ICD) interacting with a user computing device (UCD) toedit a menu selection. In this example, the coffee shop applicationdisplays an edit button. A user of the UCD selects the edit button whenthey wish to modify an item or quantity of the item previously selected.When receiving the edit selection, the UCD sends an edit signal to theICD that indicates the user wishes to edit the item and/or quantity. Theuser may the edit the menu selections by one or more of the touch screenof the UCD, the touch screen of the ICD, and a voice command.

FIG. 55 is a schematic block diagram of an example of an interactivecomputing device (ICD) interacting with a user computing device (UCD) toedit a menu selection. In this example, the menu displayed on the ICD ismirrored to the display of the UCD after the ICD receives the editsignal. Alternatively, or in addition to, the ICD screen may be sent tothe UCD for editing. The user selects the item(s) to be edited. Forexample, the user selects a quantity value button of item 2 to edit.

FIG. 56 is a schematic block diagram of an example of an interactivecomputing device (ICD) interacting with a user computing device (UCD) toedit a menu selection. After selecting the quantity of item 2, the useris prompted to enter a new value for the quantity. For example, the useris presented with an empty quantity field and digit buttons 0-9 tomanually enter a new quantity. As another example, the UCD displays thecurrent quantity with up and down arrows for the user to touch to modifythe current quantity by a default value (e.g., by 1). In this specificexample, the user edits the quantity of item 2 from a quantity of threeto a quantity of two. The ICD receives the selection and display anupdated tabulated menu (e.g., the line total for item 2 and the totalfor all items).

FIG. 57 is a schematic block diagram of an example of an interactivecomputing device (ICD) interacting with a user computing device (UCD) toedit a menu selection. As illustrated, the user selects an end editbutton to finalize editing and returns to a previous screen or to afinalize order screen. Once the editing is complete, the ICD maydiscontinue the mirroring and the UCD may return the display to the lastshown in the application.

FIG. 58 is a schematic block diagram of an example of an interactivecomputing device (ICD) interacting with a user computing device (UCD) toedit a menu selection. To finishing the editing, the user selects a donebutton, which causes the UCD to send a done signal to the ICD. In anexample, the done button causes the selected items to be ordered. In aninstance, the UCD and ICD then communicate via an STS connection (e.g.,BaaN, close proximity, etc.) to process payment for the ordered items.

FIG. 59 is a schematic block diagram of an embodiment of setting upscreen to screen (STS) communications between a user computing device(UCD) 14 and an interactive computing device (ICD) 12. The UCD 14 andthe ICD 12 both include a screen to screen (STS) communicationapplication 90.

The example begins by the UCD 14 sending (1) a user identification (ID)package 570 to the ICD via an STS connection. The user ID package 570includes a user ID information for the UCD 571, STS account IDinformation 572, and UCD ID information 573. In an example, one or moreportions of the information 571-573 is confidential information 141.

The user ID information for UCD 571 includes one or more of a user namefield, a password (PW) field, an address field, a phone number field, adate of birth (DOB) field, and a personal field. The personal data fieldincludes data that further identifies a user of the UCD (e.g., personalcodes, personal biometric data, etc.). The STS account ID information572 includes one or more of a user name field, a PW field, an account(acct) ID field, and a time stamps field. The time stamps field mayinclude times regarding creation of an STS account, a last 10 STS uses,etc.). The UCD ID information 573 includes one or more of aninternational mobile equipment identity (IMEI) field and an internetprotocol (IP) address field.

Continuing with the example, after receiving the user ID package 570,the ICD 12 creates a verification package 578, which includes anaggregate of the user ID package 570 and an ICD operator ID package 574or selected portions thereof. The ICD operator ID package 574 includesoperator ID information for the ICD 575, STS account ID information 576,and ICD ID information 577. The information 575 includes one or more ofan operator name field, an operator password field, an address field, aphone number field, and an operator unique ID field. The STS account IDinformation 576 includes an operator name field, a password field, andan account ID field. The ICD ID information 577 field includes an IMIDfield and an IP address field. In an example, the verification package578 includes a user name and password of information 571, and anoperator name and password of information 575. Note one or more portionsof the information 575-577 is classified as confidential information141.

FIG. 60 is a schematic block diagram of the example of the setting upscreen to screen (STS) communications of FIG. 59 that continues with theinteractive computing device (ICD) sending (3) the verification package578 to a screen to screen (STS) communication server 22. The STScommunication server 22 receives and reviews (4) the verificationpackage 578. The reviewing may include one or more verifications. As anexample of a first verification, the STS communication server 22 reviewsthe verification package 578 by looking up and verifying the user STSaccount information that is stored in a user database 580 substantiallymatches the user STS account information included in verificationpackage 578. As an example of a second verification, the STScommunication server 22 further reviews the verification package 578 bylooking up and verifying the ICD operator STS account information thatis stored in ICD database 582 substantially matches the ICD operator STSaccount information included in verification package 578.

Having reviewed the verification package 578, the STS communicationserver 22 sends (5) an acknowledgement (ACK) or error message to one orboth of the user computing device (UCD) 14 and the ICD 12. For example,the STS communication server 22 sends an ACK to UCD 14 when the reviewof the user STS account information is favorable (e.g., user STS accountinfo in verification package 578 substantially the same as user STSaccount information stored in user database 580). As another example,the STS communication server 22 sends an error message to ICD 12 whenthe review of the ICD operator STS account information is unfavorable(e.g., ICD operator STS account information in verification package 578is not substantially the same as ICD operator STS account informationstored in user database 580).

The example continues with the UCD 14 confirming (6) the ACK. Forexample, when the UCD is sent an ACK from the STS communication server22, the UCD sends (6) its ACK to ICD 12. Alternatively, the UCD may senda ping verification message to the ICD that indicates a favorableacknowledgement was received by the UCD in step (5).

FIG. 61 is a schematic block diagram of the example of the setting upthe screen to screen (STS) communications of FIGS. 59-60 that continueswith the STS communication server creating a one-time use interactionsecurity code. Continuing with the example, the STS communication servercreates (7) the one-time user security interaction code. The interactionsecurity code may be one or more of an alphanumerical code (e.g.,A3zv89jb3, etc.), a numerical code (e.g., 8374), a public/private keypair, data transmission information (e.g., type of encoding,transmission frequency, etc.) and a graphical code (e.g., QR code, barcode, etc.).

Having created the security interaction code, the STS communicationserver 22 sends (8 a) a first portion of the security interaction codeto the user computing device (UCD) 14 and sends (8 b) a second portionof the security interaction code to the interactive computing device(ICD) 12. As an example, the security interaction code is a numericalcode of “8374”. Thus, a first portion could be “83” and the secondportion could be “74”. Alternatively, the first portion could be “84”with a message indicating “8” is a first digit of the numerical code and“4” is a fourth digit of the numerical code, and the second portioncould be “37” with a message indicating “3” is a second digit of thenumerical code and “7” is a third digit of the numerical code.

As another example, the security interaction code is an indication ofwhich frequencies to use (e.g., 100 Hz, 20 MHz, 3 GHz, etc.) for an STScommunication. Thus, a first portion could indicate a first frequency is100 Hz and a second portion indicates a second frequency is 120 Hz. Inyet another example, the security interaction code is an indication ofbits per cycle and the type of modulation for the STS communications. Assuch, a first portion could be “4” and the second portion could be“amplitude shift keying”.

Having received the portions of the security interaction code, the UCD14 and the ICD 14 exchange their respective portions to recreate thesecurity interaction code. For example, UCD 14 sends “83” to the ICD 12,and ICD 12 sends “74” to UCD 14 such that both the UCD 14 and ICD 12recreate security code “8374”. The recreated security code may then beverified with the STS communication server in order for the STSconnection to be utilized (e.g., for an STS communication ofconfidential information). Note that in an example, steps 7-9 areperformed after both the UCD 14 and ICD 12 have been verified in step(4).

FIG. 62 is a schematic block diagram of an embodiment of the example ofthe setting up screen to screen (STS) communications of FIGS. 59-61 thatcontinues with the ICD 12 and UCD 14 selecting (10) a modality for menuinteraction. The modality includes one or more of mirroring displaysduring menu interaction, using one of the UCD or ICD for the menuinteraction, and using both the UCD and ICD for different part of themenu interaction (e.g., UCD to order, ICD for processing payment).

In an example, the selection could be encoded using a second securityinteraction code (e.g., code that specifies a type of encoding, etc.).Note the setting up the STS communication includes determining aconnection type (e.g., BaaN, close proximity, etc.) and a communicationprotocol (e.g., what data to be transmitted via STS, via Bluetooth, viaWLAN, what frequencies to use, what modulation scheme to use, how manybits per cycle, etc.).

FIG. 63 is a logic flow diagram of an example of a method of determininga menu interaction modality between an interactive computing device(ICD) and a user computing device (UCD) (the ICD and/or UCD arehereinafter referred to in this figure as a computing device). Themethod begins at step 610, where the computing device determines whetherthe interaction with the menu is performed via an ICD touch screen orvia a UCD touch screen. When the computing device determines the menuinteraction is via the ICD screen, the method continues to step 612,where the ICD displays menu options for a user touch selection.

When the menu interaction is via the UCD touch screen, the methodcontinues to step 614, where the computing device determines whether tomirror menu display data one both of the touch screens or split the menudisplay data between the UCD and the ICD touch screens.

When the computing device determines to mirror the menu display data,the method continues to step 618, where the computing device selects awireless communication means (e.g., WLAN, Bluetooth, cellular data,etc.) to be used for the mirroring. When the computing device determinesto split the screens, the method continues to step 616, where thecomputing device selects a wireless communication means (e.g., WLAN,Bluetooth, cellular data, etc.) to be used for the splitting.

FIG. 64 is a logic flow diagram of an example of a method of setting upa screen to screen (STS) communication setting between an interactivecomputing device (ICD) and a user computing device (UCD) (the ICD and/orUCD are hereinafter referred to in this figure as a computing device).The method begins at step 630, where the computing device determineswhether an STS connection is initiated by touch (e.g., body as a network(BaaN)), or by device proximity (e.g., close proximity). When the STSconnection is initiated by touch, the method continues to step 634,where the computing device determines BaaN is to be utilized as aconnection medium for setting up the STS communication settings. Whenthe STS connection is initiated by proximity, the method continues tostep 632, where the computing device determines device to device closeproximity is to be utilized as the connection medium for setting up theSTS communications.

Having setup the STS communication medium (e.g., steps 632 and 634), themethod continues with step 636, where the computing device selects adata signaling format for the STS communications. The data signalingformat includes one or more of a frequency-time pattern encoding,frequency shift keying (FSK) on selected electrodes, amplitude shiftkeying (ASK) on selected electrodes, phase shift keying on selectedelectrodes, 4 quadrature amplitude modulation on selected electrodes,FSK/ASK combination on selected electrodes, and/or other data signalingformats. In an example, one of the previous list of data signalingformats is utilized as a default data signaling format. For example, thecomputing device determines a default data signaling format for STScommunications is ASK on selected electrodes.

Having selected a data signaling format, the method continues to step637, where the computing device determines whether the STS communicationwas successful. For example, the ICD sends a ping signal in accordancewith the selected data signaling format and determines the STScommunication was successful when receiving a favorable ping back signalfrom the UCD.

When the STS communication is successful, the method continues to step638, where the computing device selects a communication path option formenu interaction. The communication path options includes one or more ofan STS connection via BaaN, an STS connection via device to device closeproximity, an ICD touch screen direct touch, Bluetooth, wireless localarea network (WLAN), and cellular data. When the STS communication isnot successful, the method continues to step 639, where the computingdevice retries setting up the STS communication or the process ends.

FIG. 65 is a schematic block diagram of an embodiment of processing ascreen to screen (STS) transaction that includes a user computing device(UCD) 14, an interactive computing device (ICD) 12, a network 15 and apayment processing server 24. The UCD 14 includes a payment application92 and the ICD 12 includes one or more payment processing applications142.

In an example of operation, the UCD 14 sends (1) payment information tothe ICD 12 via an STS connection 18. For instance, payment application92 obtains information regarding a first payment account of a user ofthe UCD 14 as the payment information. A payment account includes one ormore of a credit card account, a debit card account, a gift cardaccount, a checking account and a loyalty rewards account. The ICD 12receives the payment information and creates (2), via a paymentprocessing application 142, a payment package. The payment packageincludes at least some of the payment information and operator accountinformation (e.g., identification info, authorization info, etc.)associated with an operator (e.g., coffee shop entity) of the ICD.

The ICD 12 sends (3) the payment package to a payment processing server24 via the network(s) 15. The payment processing server 24 processes (4)the payment and sends (5) a payment response (e.g., accepted, declined)to the ICD 12 via network(s) 15. The ICD 12 then finalizes (6) theinteraction with the UCD 14 based on the payment response. For example,when a favorable payment response (e.g., accepted) was received, the ICD12 prompts a user of the UCD to finalize the interaction via a screen toscreen (STS) connection 18. For instance, the ICD displays a messageasking the user to sign (forming a body as a network (BaaN) STSconnection between the UCD and the ICD) in a particular portion of thescreen while STS communications are enabled on the UCD 14. As anotherexample, when an unfavorable payment response is received (e.g.,declined), the ICD prompts the user for a different payment methodand/or ends the transaction.

FIG. 66 is a logic flow diagram of an example of a method of processinga screen to screen (STS) transaction between a user computing device(UCD) and an interactive computing device (ICD). In the description ofthis Figure, the ICD, UCD and/or another computing device are referredto as a computing device. Note the UCD and the ICD both have access toan STS communication application and an STS user/operator interactiveapplication.

The method begins or continues with step 670, where the computing devicedetermines to set up a screen to screen (STS) connection. The methodcontinues with step 672, where the UCD sends its information and userSTS communication information to the ICD via the STS connection. Themethod continues with step 674, where the ICD validates the UCD and userinformation of a user associated with the UCD. In an example, thevalidation is performed in conjunction with a cellular data network(e.g., 5G). The cellular data provider of the cellular data networkstores information utilized in setting up an STS communicationapplication used by one or both of the ICD and the UCD.

The method continues with step 676, where the ICD determines whether theUCD and user information is valid. When the UCD and user information innot valid, the method continues to step 677, where the process ends.When the UCD and user information is valid, the method continues to step678, where the ICD provides a menu of options for selection by a user ofthe UCD. In an example, the menu of options are displayed on a touchscreen of the ICD. As another example, the menu of options are displayedon a touch screen of the UCD. As yet another example, the menu ofoptions are displayed on a display of the of UCD and the ICD (e.g.,mirrored, split (some items or menu info shown on one device but not theother). When the menu of options are displayed on the UCD, the UCD mayobtain the data needed for the displaying via an STS connection, viaBluetooth, via a wireless local area network, and/or via a cellular datanetwork.

The method continues with step 680, where the ICD receive selections ofthe menu of options from a user of the UCD. The method continues withstep 682, where the ICD determines whether the user is done selectingfrom the menu of options. The determination includes one or more of atimeout, a command, and a selection of a done button. When the user isnot done selecting, the method loops back to step 680. When the user isdone selecting, the method continues to step 684, where the ICDsummarizes the users' selections and displays an amount due for theselections and/or a line item amount for each of the individualselections.

The method continues with step 686, where the computing devicedetermines whether the UCD accepts the summarized selections. When theUCD does not accept the summarized selections, the method continues tostep 688, where the computing device determines whether to edit or endthe process. When ending the process, the method continues to step 689,where the process ends. When editing, the method continues to step 690where the UCD edits the summarized selections. For example, the UCDreceives inputs (e.g., touch, voice, eye movement, etc.) from a user ofthe UCD and interprets the inputs as edits of the previous selections.The method then continues back to step 682. Alternatively, at step 688,the UCD may go back to step 686 and accept the summarized selections(e.g., when determining the user accidently did not accept (e.g.,touched wrong button)).

When the UCD accepts the selections, the method continues with step 692,where the computing device determines whether a payment is needed. Forexample, the ICD determines whether the user of the UCD has pre-paid forthe selections or the user is paying cash. In an example, the paymentincludes store loyalty points, a gift card, and a pre-paid accountassociated with a store that the ICD has authorization to access. Whenno payment is needed, the method continues to step 702, where theinteraction is completed.

When payment is needed, the method continues to step 694, where the UCDprovides the ICD with payment information via an STS connection. Themethod continues with step 696, where the ICD processes the payment. Themethod continues with step 698, where the ICD determines whether thepayment was successfully processed. When the payment is successfullyprocessed, the method continues to step 702, to complete theinteraction.

When the payment is not successfully processed, the method continues tostep 699, where the computing device determines whether to use analternative payment method (e.g., different card, points instead ofcard, etc.) or end the transaction. When ending, the method continues tostep 700, where the process ends. When using an alternative method, themethod continues to step 694, where the UCD provides the ICD with thealternative payment information (e.g., identification of a loyaltyaccount).

FIG. 67 is a logic flow diagram of an example of a method of processinga screen to screen (STS) transaction between a user computing device(UCD) and an interactive computing device (ICD). In the description ofthis Figure, the ICD, the UCD and/or another computing device arereferred to as a computing device.

The method begins or continues with step 710, where the UCD receives amessage to initiate interaction the ICD. The method continues to step712, where the UCD identifies one or more servers (e.g., STScommunication server, interaction application server, etc.) associatedwith the ICD. The method continues with step 714, where the UCD sends aUCD verification package to the identified servers. The method continueswith step 716, where the computing device determines whether it hasreceived a transaction message. When the computing device has notreceived the transaction message, the method continues to step 717,where the process ends, is retried, an update is performed (e.g., updateuser profile, update STS interactive user application, etc.) or otheraction is performed based on the transaction message.

When the computing device has received the transaction message, themethod continues to step 718, where the computing device sets up an STSconnection between the UCD and the ICD. For example, the ICD prompts theuser to touch a portion of the touch screen of the ICD while the user istouching (holding, in pocket, etc.) the UCD. The method continues withstep 720, where the UCD sends a transaction message to the ICD via theSTS connection. The method continues with step 722, where the ICDprovides a menu of options. For example, the ICD displays the menu ofoptions on a touch screen associated with the ICD. The method continueswith step 724, where the ICD receives selections of options. The methodcontinues with step 726, where the computing device determines whether auser is done selecting. When the user is not done selecting, the methodloops back to step 724.

When the user is done selecting, the method continues to step 728, wherethe ICD summarizes the selections of the user and a total amount due forthe selections. The method continues with step 730, where the computingdevice determines whether the summarized selections are accepted by theuser of the UCD. When the summarized selections are not accepted, themethod continues to step 732, where the computing device determineswhether to edit the transaction or end. When ending, the methodcontinues to step 734, where the process ends. When editing, the methodcontinues to step 736, where the UCD edits the summarized selections.After the UCD edits the summarized selections, the method then continueswith step 724.

When the selections are accepted, the method continues to step 740,where the computing device determines whether payment is needed. Whenpayment is not needed the method continues to step 750, where theinteraction is completed. When the payment is needed, the methodcontinues to step 742, where the UCD provides the ICD paymentinformation via the STS connection. The method continues with step 744,where the ICD processes payment. The method continues with step 746,where the computing device determines whether payment was processedsuccessfully.

When the payment was not processed successfully, the method continues tostep 748, where the computing device determines whether to use analternative payment method (e.g., different card, loyalty account pointsinstead of card, etc.) or to end the transaction. When ending, themethod continues to step 749, where the process ends. When using analternative method, the method continues to step 742, where the UCDprovides the ICD with the alternative payment information (e.g.,identification of a loyalty account). When the payment was processedsuccessfully, the method continues to step 750, where the interaction iscompleted.

FIG. 68 is a schematic block diagram of an embodiment of processing ascreen to screen (STS) transaction that includes a user computing device(UCD) 14, an interactive computing device (ICD) 12, an interactionapplication server 20, a user database 580, and an interactive computingdevice (ICD) database 582. The user computing device 14 includes a userinteraction application 148 and an STS communication application 90. TheICD 12 includes an operator interaction application 140 and an STScommunication application 90. The user/operator interaction applicationis one or more of a coffee shop, a restaurant chain, a gas station,ticket sales company, a fast food chain, a department store, a hardwarestore, an auto repair store, an amusement park, a move theatre, and amusical instrument store.

In an example of operation, the UCD 14 sends (1) a user applicationidentification (ID) package to the ICD 12 via an STS connection. The ICDcreates (2) a verification package based on the user application IDpackage. The user application ID package and the verification packageare similar to the user ID package and verification package of FIG. 57,but these are specific to the interaction applications run on the UCD 14and the ICD 12. For example, user application identification packageincludes user account information and the verification package includesoperator account information and at least some of the user accountinformation. In an example, the user application identification packageis encoded and/or encrypted using an interaction security code.

Having created the verification package, the ICD sends the verificationpackage to the interaction application server 20. The interactionapplication server 20 reviews (4) the verification package to verifywhether the UCD and ICD are valid and/or authorized. For example, theinteraction application server 20 looks up and verifies (4 a) the useraccount information with the user data base 580. As another example, theinteraction application server 20 looks up and verifies (4 a) the ICDoperator account information with the ICD database 582.

Having reviewed the verification package, the interaction applicationserver 20 sends an acknowledgement or error to the ICD 12 and/or the UCD14. For example, when the interaction application server 20 determinesthe user account information of the verification package substantiallymatches user account information stored in user database 580, theinteraction application server sends an acknowledgement to the UCD 14.Alternatively, or in addition to, when the interaction applicationserver 20 determines the user account information of the verificationpackage substantially matches user account information stored in userdatabase 580, the interaction application server sends anacknowledgement to the ICD 12. As another example, when the interactionapplication server 20 determines the operator account information of theverification package does not substantially match operator accountinformation stored in ICD database 582, the interaction applicationserver sends an error to one or more of the UCD 14 and the ICD 12. Whenthe ACKs are received by both the ICD and the UCD, the ICD displays (6)a menu screen.

FIG. 69 is a schematic block diagram of an example of processing ascreen to screen (STS) transaction that continues with the example ofFIG. 68. The interaction application server 20 creates (7) a one-timeuse second security code. The interaction server sends (8 a) a firstportion of the second security code to the UCD 14 and sends (8 b) asecond portion of the second security code to the ICD 12. As a specificexample, the second security code is four numerical characters “4092”.The interaction application server 20 sends a first and third digit ofthe second security code (e.g., “4” and “9”) to the UCD 14 and sends asecond and fourth digit of the second security code (e.g., “0” and “2”)to the ICD 12.

The UCD 14 and the ICD exchange (e.g., via the STS connection, viaBluetooth, etc.) their respective portions of the second security codeso the each device can recreate (9) the second security code. Forexample, the UCD encrypts the first portion of the security code with apublic key of the ICD and sends the encrypted first portion to the ICD.The ICD decrypts the encrypted first portion using its private key torecover the first portion (e.g., “4” and “9”). The ICD then combines therecovered first portion with the second portion to recreate the securitycode of “4092”. The UCD and ICD may then verify the code with the STScommunication server to process a transaction.

FIG. 70 is a logic flow diagram of an example of a method of validatinga user computing device (UCD) and user information in part utilizing ascreen to screen (STS) connection. The logic flow diagram is accompaniedby a truth table for the UCD and an interactive computing device (ICD).

The method begins or continues with step 760, where a user computingdevice (UCD) sends a user identification package (e.g., digital data) toan interactive computing device (ICD) via the STS connection. The userID package includes one or more of user identifying information (e.g.,name, birthday, etc.), account identifying information (e.g., name,routing number, card number, etc.), and UCD identifying information(e.g., MAC address, international mobile equipment identity (IMEI), cellnumber, etc.).

The method continues with step 762, where the ICD generates averification package. The verification package includes at least some ofthe user identification package, ICD identifying information, and ICDowner/operation identifying information. The method continues with step764, where the ICD sends the verification package to the STScommunication server. The ICD may utilize a unique key pair (e.g., of apublic/private key pair, etc.) to encrypt the verification package thatit obtained earlier by registering with the STS communication server forsending packages.

The method continues with step 766, where the STS communication serverreviews the verification package. For example, the STS communicationserver determines, based on the verification package, whether a usercomputing device (UCD), a user associated with the UCD, and an STSaccount identification information of the user and/or UCD is valid (1)or invalid (e.g., 0). When any values shown in the truth table include azero (0), the STS communication server determines the verificationpackage is invalid. When the values only include ones (1), the STScommunication server determines the verification package for the user isvalid.

As another example, the STS communication server determines, based onthe verification package, whether an interactive computing device (ICD),an operator associated with the ICD, and an STS account identificationinformation of the operator and/or ICD is valid (1) or invalid (e.g.,0). When any values shown in the truth table include a zero (0), the STScommunication server determines the verification package is invalid.When the values only include ones (1), the STS communication serverdetermines the verification package for the operator is valid.

The method continues with step 768, where the computing devicedetermines whether the verification package was successfully verified.When the verification package was successfully verified, the methodcontinues to step 770, where the STS communication server sends anacknowledgement (ACK) to the ICD. When the verification package was notsuccessfully verified, the method continues to step 769, where the STScommunication server sends an invalid STS setup message to the ICD.

FIG. 71 is a logic flow diagram of an example of a method of setting upa screen to screen (STS) connection between a user computing device(UCD) and an interactive computing device (ICD). In the description ofthis Figure, the ICD, the UCD and/or another computing device arereferred to as a computing device. The method begins or continues withstep 780, where the UCD sends an STS communication verification to theSTS communication server. The method concurrently continues with steps782 and 790. At step 790, the ICD sends the STS communicationverification (e.g., as a verification package) to an STS communicationserver. The method continues with step 792, where the ICD determineswhether it has received an acknowledgment from the STS communicationserver. When the ICD has not received the acknowledgment, the methodcontinues back to step 790. Alternatively, the method times out and endswhen the ICD has not received the ACK within a time period. At step 782,the UCD determines whether it has received an acknowledgement (ACK) fromthe STS communication server. When the UCD has not received the ACK, themethod continues back to step 780. Alternatively, the method times outand ends when the ICD has not received the ACK within a time period.

When the ICD and the UCD have received the acknowledgment, the methodcontinues to step 784, where the ICD and UCD exchange their respectivelyacknowledgments. The method concurrently continues with steps 786 and794, where the UCD and ICD both determine whether both acknowledgementsare favorable.

When the UCD determines that both ACKs are not favorable, the methodcontinues to step 780. When the UCD determines that both ACKs arefavorable, the method continues to step 788, where the UCD sends aninteractive user application verification to an interactive applicationserver. The method continues with step 800, where the UCD determineswhether it has received a positive (e.g., favorable, indicationverification passed, etc.) acknowledgement (ACK) from the interactiveapplication server. When the UCD has not received a positiveacknowledgement (ACK), the method continues back to step 780. When theUCD has received a positive acknowledgement (ACK), the method continuesto step 796.

When the ICD determines that both ACKs are not favorable, the methodcontinues to step 790. When the ICD determines that both ACKs arefavorable, the method continues to step 796. At step 796, the UCD andICD setup a STS communication link (e.g., connection) for an interactiveuser transaction. For example, the UCD and ICD use a body as a network(BaaN) STS connection to communicate payment information to complete atransaction.

It is noted that terminologies as may be used herein such as bit stream,stream, signal sequence, etc. (or their equivalents) have been usedinterchangeably to describe digital information whose contentcorresponds to any of a number of desired types (e.g., data, video,speech, text, graphics, audio, etc. any of which may generally bereferred to as ‘data’).

As may be used herein, the terms “substantially” and “approximately”provides an industry-accepted tolerance for its corresponding termand/or relativity between items. For some industries, anindustry-accepted tolerance is less than one percent and, for otherindustries, the industry-accepted tolerance is 10 percent or more. Otherexamples of industry-accepted tolerance range from less than one percentto fifty percent. Industry-accepted tolerances correspond to, but arenot limited to, component values, integrated circuit process variations,temperature variations, rise and fall times, thermal noise, dimensions,signaling errors, dropped packets, temperatures, pressures, materialcompositions, and/or performance metrics. Within an industry, tolerancevariances of accepted tolerances may be more or less than a percentagelevel (e.g., dimension tolerance of less than +/−1%). Some relativitybetween items may range from a difference of less than a percentagelevel to a few percent. Other relativity between items may range from adifference of a few percent to magnitude of differences.

As may also be used herein, the term(s) “configured to”, “operablycoupled to”, “coupled to”, “connected”, and/or “coupling” includesdirect coupling between items and/or indirect coupling between items viaan intervening item (e.g., an item includes, but is not limited to, acomponent, an element, a circuit, and/or a module) where, for an exampleof indirect coupling, the intervening item does not modify theinformation of a signal but may adjust its current level, voltage level,and/or power level. As may further be used herein, inferred coupling(i.e., where one element is coupled to another element by inference)includes direct and indirect coupling between two items in the samemanner as “coupled to”.

As may even further be used herein, the term “configured to”, “operableto”, “coupled to”, or “operably coupled to” indicates that an itemincludes one or more of power connections, input(s), output(s), etc., toperform, when activated, one or more its corresponding functions and mayfurther include inferred coupling to one or more other items. As maystill further be used herein, the term “associated with”, includesdirect and/or indirect coupling of separate items and/or one item beingembedded within another item.

As may be used herein, the term “compares favorably”, indicates that acomparison between two or more items, signals, etc., provides a desiredrelationship. For example, when the desired relationship is that signal1 has a greater magnitude than signal 2, a favorable comparison may beachieved when the magnitude of signal 1 is greater than that of signal 2or when the magnitude of signal 2 is less than that of signal 1. As maybe used herein, the term “compares unfavorably”, indicates that acomparison between two or more items, signals, etc., fails to providethe desired relationship.

As may be used herein, one or more claims may include, in a specificform of this generic form, the phrase “at least one of a, b, and c” orof this generic form “at least one of a, b, or c”, with more or lesselements than “a”, “b”, and “c”. In either phrasing, the phrases are tobe interpreted identically. In particular, “at least one of a, b, and c”is equivalent to “at least one of a, b, or c” and shall mean a, b,and/or c. As an example, it means: “a” only, “b” only, “c” only, “a” and“b”, “a” and “c”, “b” and “c”, and/or “a”, “b”, and “c”.

As may also be used herein, the terms “processing module”, “processingcircuit”, “processor”, “processing circuitry”, and/or “processing unit”may be a single processing device or a plurality of processing devices.Such a processing device may be a microprocessor, micro-controller,digital signal processor, microcomputer, central processing unit, fieldprogrammable gate array, programmable logic device, state machine, logiccircuitry, analog circuitry, digital circuitry, and/or any device thatmanipulates signals (analog and/or digital) based on hard coding of thecircuitry and/or operational instructions. The processing module,module, processing circuit, processing circuitry, and/or processing unitmay be, or further include, memory and/or an integrated memory element,which may be a single memory device, a plurality of memory devices,and/or embedded circuitry of another processing module, module,processing circuit, processing circuitry, and/or processing unit. Such amemory device may be a read-only memory, random access memory, volatilememory, non-volatile memory, static memory, dynamic memory, flashmemory, cache memory, and/or any device that stores digital information.Note that if the processing module, module, processing circuit,processing circuitry, and/or processing unit includes more than oneprocessing device, the processing devices may be centrally located(e.g., directly coupled together via a wired and/or wireless busstructure) or may be distributedly located (e.g., cloud computing viaindirect coupling via a local area network and/or a wide area network).Further note that if the processing module, module, processing circuit,processing circuitry and/or processing unit implements one or more ofits functions via a state machine, analog circuitry, digital circuitry,and/or logic circuitry, the memory and/or memory element storing thecorresponding operational instructions may be embedded within, orexternal to, the circuitry comprising the state machine, analogcircuitry, digital circuitry, and/or logic circuitry. Still further notethat, the memory element may store, and the processing module, module,processing circuit, processing circuitry and/or processing unitexecutes, hard coded and/or operational instructions corresponding to atleast some of the steps and/or functions illustrated in one or more ofthe Figures. Such a memory device or memory element can be included inan article of manufacture.

One or more embodiments have been described above with the aid of methodsteps illustrating the performance of specified functions andrelationships thereof. The boundaries and sequence of these functionalbuilding blocks and method steps have been arbitrarily defined hereinfor convenience of description. Alternate boundaries and sequences canbe defined so long as the specified functions and relationships areappropriately performed. Any such alternate boundaries or sequences arethus within the scope and spirit of the claims. Further, the boundariesof these functional building blocks have been arbitrarily defined forconvenience of description. Alternate boundaries could be defined aslong as the certain significant functions are appropriately performed.Similarly, flow diagram blocks may also have been arbitrarily definedherein to illustrate certain significant functionality.

To the extent used, the flow diagram block boundaries and sequence couldhave been defined otherwise and still perform the certain significantfunctionality. Such alternate definitions of both functional buildingblocks and flow diagram blocks and sequences are thus within the scopeand spirit of the claims. One of average skill in the art will alsorecognize that the functional building blocks, and other illustrativeblocks, modules and components herein, can be implemented as illustratedor by discrete components, application specific integrated circuits,processors executing appropriate software and the like or anycombination thereof.

In addition, a flow diagram may include a “start” and/or “continue”indication. The “start” and “continue” indications reflect that thesteps presented can optionally be incorporated in or otherwise used inconjunction with one or more other routines. In addition, a flow diagrammay include an “end” and/or “continue” indication. The “end” and/or“continue” indications reflect that the steps presented can end asdescribed and shown or optionally be incorporated in or otherwise usedin conjunction with one or more other routines. In this context, “start”indicates the beginning of the first step presented and may be precededby other activities not specifically shown. Further, the “continue”indication reflects that the steps presented may be performed multipletimes and/or may be succeeded by other activities not specificallyshown. Further, while a flow diagram indicates a particular ordering ofsteps, other orderings are likewise possible provided that theprinciples of causality are maintained.

The one or more embodiments are used herein to illustrate one or moreaspects, one or more features, one or more concepts, and/or one or moreexamples. A physical embodiment of an apparatus, an article ofmanufacture, a machine, and/or of a process may include one or more ofthe aspects, features, concepts, examples, etc. described with referenceto one or more of the embodiments discussed herein. Further, from figureto figure, the embodiments may incorporate the same or similarly namedfunctions, steps, modules, etc. that may use the same or differentreference numbers and, as such, the functions, steps, modules, etc. maybe the same or similar functions, steps, modules, etc. or differentones.

Unless specifically stated to the contra, signals to, from, and/orbetween elements in a figure of any of the figures presented herein maybe analog or digital, continuous time or discrete time, and single-endedor differential. For instance, if a signal path is shown as asingle-ended path, it also represents a differential signal path.Similarly, if a signal path is shown as a differential path, it alsorepresents a single-ended signal path. While one or more particulararchitectures are described herein, other architectures can likewise beimplemented that use one or more data buses not expressly shown, directconnectivity between elements, and/or indirect coupling between otherelements as recognized by one of average skill in the art.

The term “module” is used in the description of one or more of theembodiments. A module implements one or more functions via a device suchas a processor or other processing device or other hardware that mayinclude or operate in association with a memory that stores operationalinstructions. A module may operate independently and/or in conjunctionwith software and/or firmware. As also used herein, a module may containone or more sub-modules, each of which may be one or more modules.

As may further be used herein, a computer readable memory includes oneor more memory elements. A memory element may be a separate memorydevice, multiple memory devices, or a set of memory locations within amemory device. Such a memory device may be a read-only memory, randomaccess memory, volatile memory, non-volatile memory, static memory,dynamic memory, flash memory, cache memory, a quantum register or otherquantum memory and/or any other device that stores data in anon-transitory manner. Furthermore, the memory device may be in a form asolid-state memory, a hard drive memory or other disk storage, cloudmemory, thumb drive, server memory, computing device memory, and/orother non-transitory medium for storing data. The storage of dataincludes temporary storage (i.e., data is lost when power is removedfrom the memory element) and/or persistent storage (i.e., data isretained when power is removed from the memory element). As used herein,a transitory medium shall mean one or more of: (a) a wired or wirelessmedium for the transportation of data as a signal from one computingdevice to another computing device for temporary storage or persistentstorage; (b) a wired or wireless medium for the transportation of dataas a signal within a computing device from one element of the computingdevice to another element of the computing device for temporary storageor persistent storage; (c) a wired or wireless medium for thetransportation of data as a signal from one computing device to anothercomputing device for processing the data by the other computing device;and (d) a wired or wireless medium for the transportation of data as asignal within a computing device from one element of the computingdevice to another element of the computing device for processing thedata by the other element of the computing device. As may be usedherein, a non-transitory computer readable memory is substantiallyequivalent to a computer readable memory. A non-transitory computerreadable memory can also be referred to as a non-transitory computerreadable storage medium.

While particular combinations of various functions and features of theone or more embodiments have been expressly described herein, othercombinations of these features and functions are likewise possible. Thepresent disclosure is not limited by the particular examples disclosedherein and expressly incorporates these other combinations.

1. A method comprising: generating, by a first computing device, asignal having an oscillating component; driving, by the first computingdevice, the signal on a first touch sense element of the first computingdevice; detecting, by the first computing device, a touch on the firsttouch sense element based on the signal; while the touch is detected,modulating, by the first computing device, the signal with data toproduce a modulated data signal; receiving, by a second computingdevice, the modulated data signal via a transmission medium and a secondtouch sense element of the second computing device, wherein thetransmission medium includes at least one of a human body and a closeproximity between the first and second computing devices; anddemodulating, by the second computing device, the modulated data signalto recover the data.
 2. The method of claim 1, wherein when the humanbody is the transmission medium, the method further comprises:detecting, by the second computing device, a second touch on the secondtouch sense element.
 3. The method of claim 1 further comprises: thefirst touch sense element includes one or more first electrodes; and thesecond touch sense element includes one or more second electrodes. 4.The method of claim 1, wherein the detecting the touch comprises:determining a change in a self-capacitance associated with the firsttouch sense element; and interpreting the change in the self-capacitanceto indicate the touch.
 5. The method of claim 1, wherein the detectingthe touch comprises: determining a change in a mutual-capacitanceassociated with the first touch sense element; and interpreting thechange in the mutual-capacitance indicates the touch.
 6. The method ofclaim 1 further comprising: generating, by second computing device, asecond signal having a second oscillating component; driving, by thesecond computing device, the second signal on the second touch senseelement of the second computing device; detecting, by the secondcomputing device, a second touch on the second touch sense element basedon the second signal; while the second touch is detected, modulating, bythe second computing device, the second signal with second data toproduce a second modulated data signal; receiving, by the firstcomputing device, the second modulated data signal via the transmissionmedium and the first touch sense element; and demodulating, by the firstcomputing device, the second modulated data signal to recover the seconddata.
 7. The method of claim 6, wherein the modulating the second signalwith second data comprises one of: backscattering the second data withthe modulated data signal to produce the second data modulated signal;and mixing the second data with the second signal, wherein the seconddata includes a second oscillating component having a second frequency.8. The method of claim 1 further comprises: the oscillating component ofthe signal having a first frequency; and the modulating the signal withthe data to produce the modulated data signal includes mixing the signalwith the data, wherein the data includes a second oscillating componenthaving a second frequency.
 9. A computer readable storage devicecomprises: a first memory section for storing operational instructionsthat, when executed by a first computing device, cause the firstcomputing device to: generate a signal having an oscillating component;drive the signal on a first touch sense element of the first computingdevice; detect a touch on the first touch sense element based on thesignal; while the touch is detected, modulate the signal with data toproduce a modulated data signal; and a second memory section for storingoperational instructions that, when executed by a second computingdevice, cause the second computing device to: receive the modulated datasignal via a transmission medium and a second touch sense element of thesecond computing device, wherein the transmission medium includes atleast one of a human body and a close proximity between the first andsecond computing devices; and demodulate the modulated data signal torecover the data.
 10. The computer readable storage device of claim 9,wherein the second memory section stores further operationalinstructions that, when the human body is the transmission medium, causethe second computing device to: detect a second touch on the secondtouch sense element.
 11. The computer readable storage device of claim 9further comprises: the first touch sense element includes one or morefirst electrodes; and the second touch sense element includes one ormore second electrodes.
 12. The computer readable storage device ofclaim 9, wherein the first memory section stores further operationalinstructions that cause the first computing device to detect the touchby: determining a change in a self-capacitance associated with the firsttouch sense element; and interpreting the change in the self-capacitanceto indicate the touch.
 13. The computer readable storage device of claim9, wherein the first memory section stores further operationalinstructions that cause the first computing device to detect the touchby: determining a change in a mutual-capacitance associated with thefirst touch sense element; and interpreting the change in themutual-capacitance indicates the touch.
 14. The computer readablestorage device of claim 9 further comprising: a third memory section forstoring operational instructions that, when executed by the secondcomputing device, cause the second computing device to: generate asecond signal having a second oscillating component; drive the secondsignal on the second touch sense element of the second computing device;detect a second touch on the second touch sense element based on thesecond signal; while the second touch is detected, modulate the secondsignal with second data to produce a second modulated data signal;receive the second modulated data signal via the transmission medium andthe first touch sense element; and demodulate the second modulated datasignal to recover the second data.
 15. The computer readable storagedevice of claim 14, wherein the second memory section stores furtheroperational instructions that cause the second computing device tomodulate the second signal with second data by one of: backscatteringthe second data with the modulated data signal to produce the seconddata modulated signal; and mixing the second data with the secondsignal, wherein the second data includes a second oscillating componenthaving a second frequency.
 16. The computer readable storage device ofclaim 9 further comprises: the oscillating component of the signalhaving a first frequency; and the modulating the signal with the data toproduce the modulated data signal includes mixing the signal with thedata, wherein the data includes a second oscillating component having asecond frequency.